Method and apparatus for presentation of digital content

ABSTRACT

Methods and apparatus for interacting with a tag in a radio target area. More specifically, the present invention relates to methods and systems for monitoring temperature and other environmental conditions in a storage area and displaying environmental conditions as digital content in a user interactive interface based upon energy levels received from a radio target area and content from a sensor generating digital content.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to the Non-Provisional patentapplication Ser. No. 17/374,225, filed Jul. 13, 2021, and entitledMETHOD AND APPARATUS FOR INTERACTING WITH A NODE IN A STORAGE AREA,which is a continuation in part application of Ser. No. 17/134,824 filedDec. 28, 2020 and entitled METHOD AND APPARATUS FOR INTERACTING WITH ATAG IN A COLD STORAGE AREA as a continuation in part application, whichin turn claims priority Non-Provisional patent application Ser. No.16/943,750 filed Jul. 30, 2020 and entitled COLD STORAGE ENVIRONMENTALCONTROL AND PRODUCT TRACKING as a continuation in pan application, whichin turn claims priority to the Non-Provisional patent application Ser.No. 16./915,15,5 filed Jun. 29, 2020 and entitled METHOD OF WIRELESSDETERMINATION OF A POSITION OF A NODE as a continuation in partapplication, which in turn claims priority to the Non Provisional PatentApplication Ser. No. 16/775,223, filed Jan. 28, 2020 and entitledSPATIAL SELF-VERIFYING ARRAY OF NODES as a continuation in part; whichin turn claims priority to the Non Provisional patent application Ser.No. 16/688,775, filed Nov. 19, 2019 and entitled METHOD AND APPARATUSFOR WIRELESS DETERMINATION OF POSITION AND ORIENTATION OF A SMART DEVICEas a continuation in part and to Non Provisional patent application Ser.No. 16/721,906, filed. Dec. 19, 2019 and entitled METHOD AND APPARATUSFOR DETERMINING A DIRECTION OF INTEREST as a continuation in part; thepresent application also claims priority to the Non-Provisional patentapplication Ser. No. 16/900,753 filed Jun. 12, 2020 and entitled METHODAND APPARATUS FOR AUTOMATED SITE AUGMENTATION as a continuation in partapplication; and to the Non-Provisional patent application Ser. No.16/898,602 filed Jun. 11, 2020 and entitled METHOD AND APPARATUS FORENHANCED POSITION AND ORIENTATION DETERMINATION as a continuationapplication, which in turn claims priority to Non-Provisional patentapplication Ser. No. 16/817,926 filed Mar. 13, 2020 and entitledAPPARATUS FOR OPERATION OF CONNECTED INFRASTRUCTURE as a continuation inpart application; and to Non-Provisional patent application Ser. No.16/905,048 filed Jun. 18, 2020 and entitled APPARATUS FOR DETERMINING ADIRECTION OF INTEREST as a continuation application; and toNon-Provisional patent application Ser. No. 16/503,878 filed Jul. 05,2019 and entitled METHOD AND APPARATUS FOR ENHANCED AUTOMATED WIRELESSORIENTEERING as a continuation in part application: and toNon-Provisional patent application Ser. No. 16/142,275 filed Sep. 26,2018 and entitled METHODS AND APPARATUS FOR ORIENTEERING as acontinuation in part application, which in turn claims priority toProvisional application Ser. No. 62/769,133 filed Nov. 19, 2018 andentitled METHODS AND APPARATUS FOR ORIENTEERING. The contents of each ofthe heretofore claimed matters are relied upon and incorporated hereinby reference.

INCORPORATION BY REFERENCE TO RELATED APPLICATIONS

This application references the Non Provisional U.S. patent applicationSer. No. 16/504,919, filed Jul. 8, 2019 and entitled METHOD ANDAPPARATUS FOR POSITION BASED QUERY WITH AUGMENTED REALITY HEADGEAR, theentire contents of which are hereby incorporated by reference. Thisapplication references the Non Provisional patent application Ser. No.16/297,383, filed Jul. 5, 2019 and entitled SYSTEM FOR CONDUCTING ASERVICE CALL WITH ORIENTEERING, the entire contents of which are herebyincorporated by reference. This application references the NonProvisional patent application Ser. No. 16/249,574, filed Jan. 16, 2019and entitled ORIENTEERING SYSTEM FOR RESPONDING TO AN EMERGENCY IN ASTRUCTURE, the entire contents of which are hereby incorporated byreference. This application references the Non Provisional patentapplication Ser. No. 16/176,002, tiled Oct. 31, 2018 and entitled SYSTEMFOR CONDUCTING A SERVICE CALL WITH ORIENTEERING, the entire contents ofwhich are hereby incorporated by reference. This application referencesthe Non Provisional patent application Ser. No. 16/171,593, filed Oct.26, 2018 and entitled SYSTEM FOR HIERARCHICAL ACTIONS BASED UPONMONITORED BUILDING CONDITIONS, the entire contents of which are herebyincorporated by reference. This application references the NonProvisional patent application Ser. No. 16/165,517, filed Oct. 19, 2018and entitled BUILDING VITAL CONDITIONS MONITORING, the entire contentsof which are hereby incorporated by reference. This applicationreferences the Non Provisional patent application Ser. No. 16/161,823,filed Oct. 16, 2018 and entitled BUILDING MODEL WITH CAPTURE OF AS BUILTFEATURES AND EXPERIENTIAL DATA, the entire contents of which are herebyincorporated by reference. This application references the NonProvisional patent application Ser. No. 15/887,637, filed Feb. 2, 2018and entitled BUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES ANDEXPERIENTIAL DATA, the entire contents of which are hereby incorporatedby reference. This application references the Non Provisional patentapplication Ser. No. 15/716,133, filed Sep. 26, 2017 and entitledBUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVEPERFORMANCE TRACKING, the entire contents of which are herebyincorporated by reference. This application references the NonProvisional patent application Ser. No. 15/703,310, filed Sep. 5, 2017and entitled BUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURESAND OBJECTIVE PERFORMANCE TRACKING, the entire contents of which arehereby incorporated by reference. This application references the NonProvisional patent application Ser. No. 16/528,104, filed Jul. 31. 2019and entitled SMART CONSTRUCTION WITH AUTOMATED DETECTION OF ADVERSESTRUCTURE CONDITIONS AND REMEDIATION, the entire contents of which arehereby incorporated by reference. This application references theNon-Provisional U.S. patent application Ser. No. 16/657,660, filed Oct.18, 2019 and entitled METHOD AND APPARATUS FOR CONSTRUCTION ANDOPERATION OF CONNECTED INFRASTRUCTURE, the entire contents of which arehereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to storage areas with automatedenvironmental condition tracking and control. More specifically, thepresent invention relates to methods and apparatus for quantifyingthermal conditions and other environmental variables in a cold storagefacility and tracking the locations of products in the cold storagefacility and environmental conditions experienced by the product duringthe product's cold storage life cycle.

BACKGROUND OF THE DISCLOSURE

Cold storage facilities are used in a variety of industries withdiffering needs and product storage life cycles. Cold storage points mayvary depending on a type of goods to be stored and a length of a storageperiod. For example, storage temperatures may be as high as about 5°Cfor non-frozen goods such as fresh meats, dairy, fresh produce andflowers that are stored for relatively short periods of time. Frozenmeat and seafood may be stored at temperatures ranging between—10 to18°C; and industrial storage, or blood, and other biological suppliesmay need to be stored at less than −30°C.

Each temperature range and type of good may have unique needs that mayaffect how a cold storage facility is optimally cooled and how importantcontrol of other environmental factors, such as moisture, light, andvibration, may be to maintaining product quality during storage.

Accurate, uniform, and efficient control of the environmental conditionsin cold storage facilities may be difficult to achieve, particularly inlarge facilities. Without such control, the quality of goods may benegatively affected. Additionally, due to lack of uniformity inenvironmental control, it may be difficult to assess which goods havesuffered a decrease in product quality. As such, there remains a needfor improved control and monitoring of environmental conditions in coldstorage facilities.

SUMMARY OF THE DISCLOSURE

Accordingly, the present invention provides apparatus and methods forquantifying and alerting a user as to what conditions exist in a coldstorage facility and what goods or other items are present in the coldstorage facility and where respective items are located and whatconditions the items experienced while in the cold storage facility.

Locations are determined based upon wireless communications betweentransceivers located within specified areas of the cold storagefacility. The wireless communications may indicate a location of acondition quantified b a automated sensor. Quantified conditions mayinclude, for example, one or more of: a temperature currently present inthe facility, a temperature experienced by an item stored in thefacility, conditions of air introduced by an HAVC; humidity, barometricpressure, air flow, door position, presence of humans in the coldstorage space, presence of a pallet, presence of a particular item,length of presence in the facility; whether an item has been dropped orotherwise jarred; location of an item, pallet or person; and a locationand position of almost any condition quantifiable via electronic sensor.Notifications and/or alerts may be transmitted indicating one or both ofa condition present and a location of the condition.

Provided according to embodiments of the invention, are methods,apparatus, devices, and systems for tracking products throughout thestorage life cycle at a cold storage facility. Tracking includes bothlocation tracking and tracking of the environmental conditions in whichthe product has been stored at each location in the facility.

Further provided according to embodiments of the invention are systemsand methods for control of environmental conditions within a coldstorage facility, in some embodiments, a three-dimensional (3D) profileof temperature, humidity, and/or other environmental conditions may becreated and analyzed over time. The information regarding environmentalconditions over time may be correlated with the information regardingthe location of products over time to determine the environmentalconditions of a particular product throughout its time in the coldstorage facility. Additionally, monitoring changes in the 3Denvironmental profiles in the facility over time may allow foroptimization of cooling or heating cycles, air input and extraction, andother HVAC operations in view of the specific, events occurring at thefacility, the specific configuration of the facility, and the type ofproducts in the facility. For example, changes in temperature may becorrelated with specific actions at certain locations, and the systemmay use this information to optimize heating or cooling in the facility,either automatically or with user input.

Further provided according to embodiments of the invention are methodsof tracking and locating the facility's employees, infrastructure,equipment, and sensors. This tracking combined with orienteeringmethodology and devices may allow for facile location of persons orobjects in the facility. Additionally, in the event of an adverse eventwith such persons or objects, appropriate aid or resources may be routedto the appropriate location using the devices and methodology herein.

Also provided according to embodiments of the invention are userinterfaces, devices, systems, and apparatus for tracking products,employees, infrastructure, equipment, and sensors. Such devices,systems, and apparatus may perform the methods described herein,Additionally, the devices, systems, and apparatus may be in logicalcommunication with warehouse management systems (WMS) or other databasesthat track and control processes in the cold storage facility.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, that are incorporated in and constitute apart of this specification, illustrate several embodiments of thedisclosure and, together with the description, serve to explain theprinciples of the disclosure:

FIG. 1 illustrates how an Agent with Smart Device may be located usingReference Point Transceivers according to some embodiments of theinvention.

FIG. 2 illustrates a person with wearable wireless communicationtransceivers.

FIG. 2A illustrates wearable items with transceivers and sensors.

FIG. 2B illustrates a wearable item including vest type garment withtransceivers and sensors

FIG. 2C illustrates a wearable item that is torso supported withtransceivers and sensors.

FIG. 2D illustrates wearable item including footwear with transceiversand sensors.

FIG. 2E illustrates a wearable item including headgear with transceiversand sensors.

FIG. 2F illustrates a processing plant environment with persons andsensors and transceivers.

FIG. 3A illustrates an item in cold storage with mounted wirelesscommunication transceivers.

FIG. 3B illustrates methods of orienteering by device movement.

FIGS. 4A-4D illustrate exemplary configurations of antenna arrays.

FIG. 5 is an illustration of a smart device in a smart receptacleaccording to an embodiment of the invention.

FIG. 6 is a side-view of a smart device in a smart receptacle that maybe n oved in a rocking motion according to an embodiment of theinvention.

FIG. 7 illustrates line segments that intersect various position pointsin a smart receptacle according to an embodiment of the invention.

FIG. 8 is an illustration of a smart device in a smart receptacleaccording to an embodiment of the invention.

FIG. 9 an illustration of a smart device with built in transceiversaccording to an embodiment of the invention.

FIG. 10 is a flow chart providing method steps for determining, anAgent's location.

FIG. 11 is a flow chart providing method steps for determining anAgent's location.

FIG. 12 is an illustration of the use of Augmented Virtual Models inorienteering methods.

FIGS. 13A-13E are schematics of various internal structures of aProperty that may be identified and monitored according to embodimentsof the invention.

FIG. 13F illustrates an exemplary method of a user utilizing an orientedstereoscopic sensor system to orient a direction of interest.

FIG. 14 is a flow chart with method steps for creating and visualizing avirtual tag.

FIG. 15 provides an illustration of a field of view of a Smart Device.

FIGS. 16A-16G provide illustrations of a field of view depiction of aSmart Device.

FIGS. 17A-17C illustrate additional aspects of information display.

FIG. 18 illustrates a temperature sensor mesh which may be used tocreate a 3D temperature profile in some embodiments of the invention.

FIG. 19 illustrates how a thermal imaging scan may overlay on a LIDARimage.

FIG. 20 is a flowchart providing method steps for overlaying thermalimaging with LIDAR to generate a 3D temperature profile.

FIG. 21 illustrates how temperature sensitive materials may be used toenhance thermal imaging data.

FIG. 22 illustrates methods of locating the correct delivery locationusing method embodiments of the invention.

FIG. 23 illustrates the use of physical tags, N odes. Smart Devices, andvirtual tans in tracking products upon delivery.

FIG. 24 provides an illustration of a cold storage facility according toan embodiment of the invention.

FIG. 25 provides a method for tracking product location andenvironmental history according to an embodiment of the invention.

FIG. 26 illustrates mixing of pallets according to an embodiment of theinvention.

FIG. 27 illustrates a user interface according to an embodiment of theinvention.

FIG. 28 is a schematic Of an automated controller according to anembodiment of invention.

FIGS. 29A and 29B are schematics of a wireless Node according to a anembodiment of the invention.

FIG. 30 is a block diagram of a Smart Device according to an embodimentof the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

As used in the description of the invention and the appended claims, thesingular forms “a,” “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise.

A number of embodiments of the present disclosure are described herein.While this specification contains many specific implementation details,they should not be construed as limitations on the scope of anydisclosures or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments of the present disclosure.While embodiments of the present disclosure are described herein by wayof example using several illustrative drawings, those skilled in the artwill recognize the present disclosure is not limited to the embodimentsor drawings described. It should be understood the drawings and thedetailed description thereto are not intended to limit the presentdisclosure to the form disclosed, but to the contrary, the presentdisclosure is to cover all modification, equivalents and alternativesfalling within the spirit and scope of embodiments of the presentdisclosure as defined by the appended claims.

The headings used herein are for organizational purposes only and arenot meant to be used to limit the scope of the description or theclaims. As used throughout this application, the word “may” is used in apermissive sense (i.e., meaning having the potential to), rather thanthe mandatory sense (i.e., meaning must), Similarly, the words“include”, “including”, and “includes” mean including but not limitedto. To facilitate understanding, like reference numerals have been used,where possible, to designate like elements common to the figures.

The phrases “at least one”, “one or more”, and “and/or” are open-endedexpressions that are both conjunctive and disjunctive in operation. Forexample, each of the expressions “at least one of A, 13 and C”, “atleast one of A, B, or C”. “one or more of A, B, and C”, “one or more ofA, B, or C” and “A, B, and/or C” means A alone, B alone, C alone, A andB together, A and C together, B and C together, or A, B and C together.

Glossary

“Agent” as used herein refers to a person or automation capable ofsupporting a Smart Device at a geospatial location relative to a groundplane. As used herein, the term “Cold Storage Employee” or “Employee”may be used interchangeably with Agent as a specific example, but itshould be understood that any suitable Agent may be used in lieu of anEmployee, including an automated Agent. One example of an automatedAgent is a transport such as a forklift that is used to transportproducts in the cold storage facility.

“As Built,” as used herein, includes data that quantifies details of howa specific physical structure was actually constructed, In the presentinvention, a cold storage facility may be designed and modeled in a 3Dvirtual setting. As Built data is combined with a design model in avirtual setting to generate an Augmented Virtual Model. As Built datamay reflect one or more of: fabrication of the Facility; repair;maintenance; upgrades; improvements; and work order execution associatedwith the Facility.

An “Augmented Virtual Model” or “AVM” of a Property or Structure mayinclude a CAD model (or similar virtual model) of the Property orStructure, augmented with relevant data about the Property or Structure.This data may include As Built or Experiential data about rooms,features, utilities, or other facts about the Property or Structure.

“Experiential Data,” as used herein, may be generated and entered intothe AVM virtual model of the structure. Experiential Data may includedata indicative of a factor that may be tracked and/or measured inrelation to the Facility. Experiential data is typically generated bySensors in or proximate to the Facility and may include, by way ofnon-limiting example, one or more sensors, such as: vibration Sensors(such as accelerometers and piezo electro devices); force transducers;electronic temperature sensing device (e.g. thermocouple, thermistor,resistance based, etc.); amp meters, ohmmeters, switches, motiondetectors; light wavelength capture (such as infrared temperatureprofile devices), water flow meters; air flow meters: and the like. Someexamples of Experiential Data may include: details of operation ofequipment or machinery in the Processing Facility; vibrationmeasurements; electrical current draws; machine run times, machine runrates, machine run parameters; interior and/or exterior temperatures;opening and closings of doors and windows; weight loads; preventivemaintenance; cleaning cycles; air circulation; mold contents; thermalprofiles and the like.

“Image Capture Device” as used herein refers to an apparatus forcapturing digital or analog image data. An Image Capture Device may be atwo- or three-dimensional camera and may include a charge-coupled device(known as a “CCD”).

A “Modality,” as used in conjunction with a transceiver, transmitter,and/or receiver, refers to one or both of a bandwidth of wirelesscommunication and a protocol associated with a bandwidth. By way ofnon-limiting example, a modality, as used in relation to a transceiver,transmitter, and/or receiver may include: Wi-Fi; Wi-Fi RTT, Bluetooth;UWB; ultrasonic; sonic; infrared; or other logical communication medium.

“Node” as used herein means a device including at least a processor, adigital storage, and a wireless transceiver. A “Base Node” may haveadditionally functionality including the ability to aggregate data overtime, perform controller functions, transmit data via one or morewireless modality, be powered by utility-based alternating current,and/or communicate via a hardwired medium (e.g., via ethernet cabling).

“Property,” as used herein, means a Structure/Facility or piece of realproperty to be navigated in accordance with the orienteering methods andapparatus described herein. In general, the term “Property” will referto the cold storage Facility and/or the real property on which it sits.

“Ray” as used herein refers to a straight line including a startingpoint and extending indefinitely in a direction.

“Reference Point Transceiver” as used herein means an electronic devicecapable of one or both of transmitting or receiving data, and whoseposition within a Property is known or capable of being known. Anexample of a Reference Point Transceiver is a Node.

“Smart Device” as used herein includes an electronic device including,or in logical communication with, a processor, controller, and digitalstorage, and capable of executing logical commands.

“Structure” as used herein refers to a manmade assembly of partsconnected in an ordered way. Examples of a Structure in this disclosureinclude a building that includes a Cold Storage Facility. “Cold StorageFacility” or “Facility,” as used herein, refers to one or more enclosedspaces, which may include doors, bays, vents, or other methods ofingress/egress, but are generally temperature controlled via an HVAC orother cooling system. Examples of Cold Storage Structures includevaults, lockers, rooms, warehouses, and the like. A cold storagewarehouse or Facility may be described with reference to manyembodiments herein, but it is to be understood that the same systems andmethods could be used with other cold storage Structures.

“Environmental Control System” includes at least one processing devicein Communication with an environmental sensor. In some embodiments, theprocessing device acts as or is in communication with a controller thatperforms an action based on environmental data in the Facility obtainedby a sensor, wherein such actions include creating visual displays,generating alarms, and modifying a heating or cooling cycle by thecooling system. An environmental control system may include atemperature control system, but may also include other non-temperaturesensors including humidity, light, motion, vibration, weight, volume,LIDAR, other location positioning sensors/devices. A “TemperatureControl System” is one type of “Environmental Control System,” in whichthe environmental data is temperature data.

As used herein, a “3D thermal detection apparatus” refers to anyapparatus or device that can create a 3D temperature profile. In someembodiments, the apparatus may include thermal imaging software withdepth of field parameters. Examples include LIDAR relief map withthermal infrared imaging overlaid thereon. In some cases, the 3D thermaldetection apparatus may include a series of temperature sensors, suchthat a 3D profile can be obtained by extrapolating between sensors.

As used herein, a “3D temperature profile” refers to a display orreadout that provides temperature measurements throughout a 3D space,whereby temperature can be measured on X, Y, and Z axes or using polarcoordinates. The sensitivity and mechanism of display may vary dependingon the apparatus used to create the 3D temperature profile. Other typesof environmental data may monitored in a 3D space to create 3D profiles.Examples include 3D humidity profiles, 3D air flow profiles, and 3Dradiation profiles.

Tracking Position, Direction of Interest, and Field of View

Methods of tracking position of persons, objects, equipment, sensors,and other infrastructure will first be described. Such methods, devices,and systems may be used in various embodiments described herein.

Orienteering methods and apparatus may provide accurate determination ofthe position of a Smart Device, and/or an Employee or Agent inpossession of the Smart Device, relative to a destination or feature ofinterest. In some cases, highly accurate location position may bedetermined via automated apparatus and multiple levels of increasinglyaccurate location determination. A first level may include use of a GPSdevice (i.e., within the Smart Device) providing a reading to firstidentify the location of a Property on which a Cold Storage Facilitysits (i.e., the cold storage warehouse). The GPS device may further beassociated with a high-level, general description of the Property, suchas an address, lot number, or other recorded designator.

A second level may use position transmitters located within, orproximate to, the Cold Storage Facility to execute triangulationprocesses in view of on-site location references, such as a ReferencePosition Transceiver, near field radio communication beacons at knownX-Y position reference points; line of sight with physical referencemarkers; coded via ID such as bar code, hash tag, and alphanumeric orother identifier. In some embodiments, triangulation may calculate aposition within a boundary created by the reference points to withinmillimeter range. In some embodiments, differential GPS may be used toaccurately determine a location of a Smart Device with a sub centimeteraccuracy. This location may be determined with reference to two- orthree-dimensional Cartesian or polar coordinates.

In some examples, a reference point Node may be placed in a fixedlocation and function as a transceiver of signals. For example, a Nodemay receive and transmit signals in a radio frequency band of theelectromagnetic spectrum. In a simple form, a Node may receive anincoming wireless radio frequency communication and also broadcast aradio frequency wireless communication. Radio frequencies utilized forwireless communication may include those within the electromagneticspectrum radio frequencies used in UWB, Wi-Fi and Bluetooth modalities,as well as IR, bisible and UV light as examples.

In some embodiments, a Node is operative to communicate timing data anddata generated by a sensor. Such communications may provide identifyinginformation unique to the Node, data related to the synchronization oftiming with reference point transceivers, and/or data received fromother Nodes.

A triangulation calculation of the position of a Smart Device or a Nodemay result from a system of multiple reference position Nodescommunicating timing signals to or from the Smart Device or Node.Methods of calculating positions via wireless communications may includeone or more of: RTT, RSSI, AoD, AoA, timing signal differential and thelike. Triangulation or other mathematical techniques may also beemployed in determining a location.

The process of determination of a position based upon triangulation withthe reference points may be accomplished, for example via executablesoftware interacting with a controller in a Node, a Smart Device, orserver.

Referring now to FIG. 1 , an exemplary embodiment of a cold storageEmployee with a Smart Device in a Cold Storage Facility with ReferencePosition Transceivers to assist in precise location identification isshown. Reference Position Transceivers 101-104 may be deployed in adefined area 106 to determine a location 107 of an Employee/Agent 100within or proximate to the defined area 106. Reference PositionTransceivers 101-104 may be fixed in respective locations 108 a, 108 b,108 c and 108 d and transceive in a manner suitable for a triangulationdetermination of the position of the Employee. Transceiving may occurvia wireless transmission to one or more Transceivers 105 supported bythe Employee 100. By way of non-limiting example, Transceivers 105supported by the Employee 100 may be included in, or be in logicalcommunication with, a Smart Device with Transceivers 105 able totransceive with the Reference Position Transceivers 101-104. Wirelesscommunications between the Reference Point Transceivers and one or moreTransceivers 105 supported by an Agent 100 include timing data or otherinformation useful to determine a location 107 of the Transceiver 105supported by the Agent 100, within or proximate to the defined area 106.

The Reference Position Transceivers 101-104 may include devices such asa radio transmitter, radio receiver, a light generator, or animage-recognizable device (i.e., an apparatus set out in a distinctivepattern recognizable by a camera). A radio transmitter may include arouter or other Bluetooth or other communication device for enteringinto logical communication with a controller. In some embodiments,Reference Position Transceivers 101-104 may include a Wi-Fi router thatadditionally provides access to a distributed network, such as theInternet. Cartesian coordinates (including Cartesian coordinatesgenerated relative to a GPS or other reference point), or any othercoordinate system, may be used as data that may be utilized for one ormore of: locating one or both of an Employee 100; indicating a directionof interest; and identifying a Structure or defined area 106. A radiotransmitter may include a router or other Wi-Fi device. The radiotransmitter may include transmissions on traditional Wi-Fi frequencies(300 MHz-60 GHz),, including ultra-wideband frequencies (a bandwidthabove 500 MHz, e.g. between 3.1 and 10.6 GHz or 24 GHz), some specificproducts may utilize bandwidths of between 6 to 8.5 GHz; 6 to 9 GHz, 3.5to 6,5 GHz, 6 to 8 GHz, and the like. The light generator may distributelight at human-safe intensities and at virtually any frequency known inthe art. Such frequencies include, without limitation, infrared,ultraviolet, visible, or nonvisible light. Further, the light beacon maycomprise a laser, which may transmit light at any of the aforementionedfrequencies in a coherent beam.

Positions of the Reference Point Transceivers 101-104, which are fixedin respective locations 108(a-d) within or proximate to the defined area106, define a wireless communication, area (WCA) 112. Essentially theWCA 112 defines an area in which a location of a person may be tracked.A sensor may continue to quantify a physiological state of the personsupporting an appropriate sensor whether or not the person is within theWCA 112.

In some embodiments, data quantifying a physiological state of a personmay be stored within a Node or Tag supported by the person andtransmitted via wireless communication once a Node or Tag is withinwireless communication range of a Reference Point Transceiver 101-104.The Reference Point Transceivers 101-104 may include devices capable ofwireless communication via a same modality as that utilized by theAgent-supported Transceiver 105. A radio frequency transceiver includedin one or both of the Reference Point Transceivers 101-104 and theAgent-supported Transceiver 105 may therefore transmitters and receiversoperative to communicate via wireless modalities that include, forexample; Wi-Fi, Bluetooth, Ultra-wideband (“UWB”), ultrasonic, infrared,or other communication modality capable of logical communication betweenTransceivers 101-105.

In some embodiments, a Reference Point Transceiver 101-104 may include amulti-modality transceiver, that communicates more locally via a firstmodality, such as Ultrawideband (“UWB”) Bluetooth, Wi-Fi, ANT, Zigbee,BLE, Z Wave, 6LoWPAN, Thread, Wi-Fi-ah, NFC (near field communications),Dash 7, Wireless HART or similar modality; and to a greater distance viaa second modality, such as a cellular communication modality (e.g. 3G,4G, 5G and the like), sub GHz modality, Internet Protocol modalities andthe like which may provide access to a distributed network, such as theInternet. Other modalities are also within the scope for the presentinvention.

Wireless communications between Transceivers 101-105 may engage inlogical communications to provide data capable of generating one or woreof: Cartesian coordinates, polar coordinates, vector values, AoA, AoD,RTT, RSS, a GPS position, or other data that may be utilized for one ormore of: locating one or both of an Agent 100; indicating a direction ofinterest; and identify a defined area 106.

A precise location may be determined via logical processes, such astriangulation; trilateration; and/or angle phase change; based upontiming values or other mechanism to generate a distance from one or moreantennas in the multiple Reference Point Transceivers 101-104 to one ormore antennas in an Agent-supported Transceiver(s) 105.

For example, a radio transmission or light transmission be measured andcompared from three Reference Point Transceivers 101-103. Measurementmay include, one more of; a timing value, a received transmissionstrength, received transmission amplitude, and received transmissionquality.

Other embodiments may include a device recognizable via image analysisvia a sensor, LiDAR, Image Capture Device, CCD device, and the likewhich may capture an image of three or more recognizable features. Imageanalysis may identify three or more of the recognizable features and asize ratio of the respective image captured recognizable features may beutilized to calculate a distance from each and thereby a position of theAgent 100. Similarly, a height designation may be made via triangulationusing the position identifiers as reference to a known height or areference height

A plurality of modalities may allow for increased accuracy because eachmodality may have a different degree of reliability. For example, aSmart Device may measure the timing signal transmitted by a Wi-Fi routerwithin a different error tolerance than it may measure the receipt intoa photodetector of infrared laser light This has at least two principlebenefits. First, the location calculation may, in some embodiments, be aweighted average of the location calculated from each modality. Second,outliers may be shed. For example, if the standard location calculationcomprises a weighted average of the location as calculated by fivemodalities, but one modality yields a location greater than two standarddeviations from the average computed location, then that modality maynot be considered for future weighted location calculations.

Additionally, the radio transmitters and nor transceiver in the SmartDevice may comprise multiple antennas that transmit signals in astaggered fashion to reduce noise. By way of non-limiting example, ifthere are three antennas, then the may transmit a signal in intervals of20 milliseconds. Given this rate of transmission, a detected time ofarrival may be used to determine the distance between the transmitterand the antenna (i.e., the Smart Device). Moreover, the antennas maycomprise varying lengths to accommodate desirable wavelengths. Further,dead reckoning may be used to measure location, using traditionalmethods of numerical integration.

A precise location may be determined based upon wireless transmissionsbetween Nodes, such as between an Smart Device and the ReferencePosition Transceivers Timing determinations—as well as signal qualitieslike angle of arrival, angle of departure, transmission strength,transmission noise, and transmission interruptions—may be considered ingenerating relative positions of Nodes. Additional considerations mayinclude Al and unstructured queries of transmissions between Nodes andtriangulation logic based upon a measured distance from three or moreReference Position Transceivers 101-104. For example, a radiotransmission or light emission may be measured, and timing associatedwith the radio transmission or light to determine a distance betweenNodes. Distances from three Reference Position Transceivers 101-103 maybe used to generate a position of a Node in consideration. Othermethodologies include determination of a distance from one or more Nodesand a respective angle of arrival and/or angle of departure of a radioor light transmission between the Node in consideration and another Node(Reference Point Node or dynamic position Node).

Other embodiments may include a device recognizable via analysis of animage from an Image Capture Device which may include a sensor. LiDAR,CCD device or MOS device camera. The Image Capture Device may capture animage of three or more recognizable features which may in an exampleoccupy positions of Reference Position Transceivers. Image analysis mayrecognize the identification of each of three or more of therecognizable features and a size ratio of the respective image-capturedrecognizable features (compared, in some embodiments, to a knownabsolute size of the subject) may be utilized to calculate a preciseposition. Similarly, a height designation may be made via triangulationusing the position identifiers as reference to a known height or areference height. Discernment of a physical artifact may, for example,be based upon topographical renditions of physical aspects included inthe Structure, such as those measured using LIDAR, a magnetic force,image data (or a point cloud derived from image data). A pattern on asurface may convey a reference point by a recognizable pattern (whichmay be unique to the setting). Vernier or three-dimensional structure asnon-limiting examples. A Smart Device ascertaining a physical referencemark and a distance of the Smart Device to the mark may determine arelative location in space to a coordinate system of the marks.

In some embodiments of the invention, position determination in a ColdStorage Facility or on the Property contemplates determination of ageospatial location using triangulation, trilateration, ormultilateration techniques. In some embodiments, a geospatial locationrelative to one or more known reference points is generated. Thegeospatial location in space may be referred to as having an X,Yposition indicating a planar designation (e.g. a position on a flatfloor), and a Z position (e.g. a level within a Structure, such as asecond floor) may be generated based upon indicators of distance fromreference points. Indicators of distance may include a comparison oftiming signals received from wireless references. A geospatial locationmay be generated relative to the reference points. In some embodiments,a geospatial location with reference to a larger geographic area isassociated with the reference points, however, in many embodiments, acontroller will generate a geospatial location relative to the referencepoint(s) and it is not relevant where the position is located inrelation to a greater geospatial area, In addition to these Cartesiancoordinates, polar coordinates may be used, as further described below.

A geospatial location based upon triangulation may be generated basedupon a controller receiving a measurement of angles between the positionand known points at either end of a fixed baseline. A point of ageospatial location may be determined based upon generation of atriangle with one known side and two known angles. Referring again toFIG. 1 , triangulation essentially includes determining an intersectionof three distances 109-111, each distance 109-111 calculated from areference point 101-104 to an Agent-supported device 105. The presentinvention allows for a first distance 109 to be determined based upon awireless communication in a first modality; and a second distance 109and a third distance 110 determined based upon a wireless communicationin a same or different modality as the first modality. For example, afirst distance 109 may be determined based upon a wireless communicationusing Wi-Fi; a second distance 110 may be determined based upon awireless communication using Bluetooth; and a third distance 111 may bedetermined based upon a wireless communication using ultrasoniccommunication (other combinations of same and/or different communicationmodalities are also within the scope of the present invention).

A geospatial location based upon trilateration may be generated basedupon a controller receiving wireless indicators of distance and geometryof geometric shapes, such as circles, spheres, triangles and the like.

A geospatial location based upon multilateration may be generated basedon a controller receiving a measurement of a difference in distance totwo reference positions, each reference position being associated with aknown location. Wireless signals may be available at one or more of:periodically, within determined timespans, and continually. Thedetermination of the difference in distance between two referencepositions provides multiple potential locations at the determineddistance. A controller (such as one in the Smart Device) may be used togenerate a plot of potential locations. In some embodiments, thepotential determinations generally form a curve. Specific embodimentsmay generate a hyperbolic curve.

The controller may be programmed to execute code to locate a relativelyexact position along a generated curve, which is used to generate ageospatial location. The multilateration system thereby receives asinput multiple measurements of distance to reference points, wherein asecond measurement taken to a second set of stations (which may includeone station of a first set of stations) is used to generate at secondcurve. A point of intersection of the first curve and the second curveis used to indicate a specific location.

In examples described thus far, a position of an agent or user may bedetermined as discussed. It may be equally possible for a transceiver tobe associated with a package, storage element or other physical entityas shall be described further herein, utilizing the techniques as havebeen described relating to a user or agent.

In exemplary embodiments, as described herein, the distances may betriangulated based on measurements of UWB, Wi-Fi or sub GHz strength attwo points. Transceiver signals propagate outward as a wave, ideallyaccording to an inverse square law. Ultimately, a crucial feature of thepresent invention relies on measuring relative distances between twopoints. In light of the speed of Wi-Fi waves and the real-timecomputations involved in orienteering; these computations need to be ascomputationally simple as possible. Thus, depending upon a specificapplication and mechanism for quantifying a condition or location, suchas a measurement, various coordinate systems may be desirable. Inparticular, if the Smart Device moves only in a planar direction whilethe elevation is constant, or only at an angle relative to the ground,the computation is more simple.

Cartesian coordinates (including Cartesian coordinates generatedrelative to a GPS or other reference point), or any other coordinatesystem, may be used as data that may be utilized for one or more of:locating one or both of an Agent 100; indicating a direction ofinterest; and identifying a defined area 106. A radio transmitter mayinclude a router or other Wi-Fi device. The radio transmitter mayinclude transmissions via a Ultra-Wideband (“UWB”) frequenciesincluding, for example, frequencies above 500 MHz, 3.5-6.5 GHz or otherfrequencies usable with a UWB modality; on Wi-Fi frequencies (300 MHz-60GHz), sub GHz frequencies or other modality. A light generator maydistribute light at human-safe intensities and at virtually anyfrequency known in the art. Such frequencies may include, withoutlimitation: infrared, ultraviolet, visible, or nonvisible light.Further, a light beacon may comprise a laser, which may transmit lightat any of the aforementioned frequencies in a coherent beam.

One exemplary coordinate system includes a polar coordinate system. Oneexample of a three-dimensional polar coordinate system is a sphericalcoordinate system. A spherical coordinate system typically comprisesthree coordinates: a radial coordinate, a polar angle, and an azimuthalangle (r, θ, and φ, respectively, though θ and φ are occasionallyswapped conventionally).

By way of non-limiting example, suppose Point 1 is considered the originfor a spherical coordinate system (i.e., the point (0, 0. 0)). EachWi-Fi emitter e-1, e2, e3 can be described as points (r1, θ1, φ1), (r2,θ2, φ2), and (r3, θ3, φ3), respectively. Each of the ri's (1<i<3)represent the distance between the Wi-Fi emitter and the Wi-Fi receiveron the Smart Device.

It is understood that in some embodiments, an azimuth may include anangle, such as a horizontal angle determined in an arcuate manner from areference plane or other base direction line, such as an angle formedbetween a reference point or reference direction; and line (ray orvector) such as a ray or vector generated from or continuing to a SmartDevice. In preferred embodiments, the ray or vector may be generallydirected from a Reference Position Transceiver towards, and/or intersectone or more of: an item of interest; a point of interest; anarchitectural aspect (such as a wall, beam, header, corner, arch,doorway, window, etc.); an installed component that may act as areference in an augmented virtual model (AVM) (such as, for example, anelectrical outlet, a light fixture, a plumbing fixture, an architecturalaspect; an item of equipment; an appliance; a multimedia device, etc.);another Reference Position Transceiver or Other identifiabledestination.

Accordingly, in some embodiments, a spherical coordinate system mayinclude Reference Position Transceiver that is capable of determining anangle of departure of a location signal and a Transceiver that iscapable of determining an angle of arrival of the location signal; oneor both of which may be used to facilitate determination of anapplicable azimuth.

According to various embodiments of the present invention, one or bothof an angle of departure and an angle of arrival may therefore beregistered by a Transceiver that is transmitting and/or receivingwireless signals (e.g. radio frequency, Bluetooth 5.1, sonic frequency,or light. frequency).

In another, aspect, multiple timing signals that have been wirelesslycommunicated may be mathematically processed to increase an overallaccuracy, such as, for example, combined into a weighted average,average, mean, weighted mean. In situations with multiple timing signalscommunicated via different modalities a weighted average may bedetermined to be most beneficial. In some embodiments, conditions in anenvironment through which the wireless communication of values forvariables enabling determination of a location of a Transceiver 105 maybe quantified by sensors and the quantified conditions may be used tofurther ascertain a beneficial mathematical process. For example, ifsensors indicate significant electrical interference in a particularbandwidth of the electromagnetic spectrum, such as a spectrum utilizedby UWB and/or Bluetooth modalities, a mathematical process may give ahigher weight to a value (for a variable useful in determining theposition of a Transceiver 105) transmitted via an infrared modalitywhich is not effected by the electrical interference. Likewise, if asensor reading quantifies a significant amount of particulate or otherinterference in an atmosphere through which a wireless communication istransmitted, a lower mathematical weight may be allocated to an infraredtransmission (or other light beam effected by particulate) value.

Also, processing may allow for outliers of values for variables usefulin determining a location to be shed. For example, if a standardlocation calculation comprises a weighted average of multiple values forvariable useful for determining a location, which may include valuesgenerated based upon wireless communication using multiple modalities,but a particular modality yields a location greater than two standarddeviations from an average computed location, then that modality may notbe considered a recalculation and/or in future weighted locationcalculations. Similarly, values generated using a single modality thatfall outside a designated deviation (e.g. two standard deviations orthree standard deviations) may be excluded from a value generated viamathematical processing (e.g. average, weighted average, mean, median,mode, etc.).

Additionally, the radio transmitters, receivers, and/or transceivers ina Tag, Node and/or Smart Device may include multiple antennas thattransmit and/or receive electromagnetic transmissions. In someembodiments, the multiple antennas may transmit and/or receive in astaggered fashion suitable to reduce noise.

Additional considerations may include AI and unstructured queries oftransmissions between Tags and triangulation logic based upon a measureddistance from three or more Reference Point Transceivers 101-104. Forexample, a radio transmission or light emission may be measured, andtiming associated with the radio transmission or light emission todetermine a distance between disparate transceivers included in Tags.Nodes, Smart Devices and the like. Distances from three Reference PointTransceivers 101-103 may be used to generate a position of a Transceiverin consideration. Other methodologies include determination of adistance from one or more Transceiver and a respective angle of arrivaland/or angle of departure of a radio or light transmission between theNode in consideration and another Transceiver (Reference PointTransceiver or dynamic position Transceiver, e.g. an Agent supported lagor Node).

In some embodiments, locating an Agent occurs in or proximate to a ColdStorage Facility in which Reference Position Transceivers, (including,for example, one or more of: Wi-Fi Transceivers, UWB Transceivers,Bluetooth Transceivers, infrared Transceivers and ultrasonicTransceivers) may be located above and/or below an Employee. In theseembodiments, a cylindrical coordinate system may be more appropriate. Acylindrical coordinate system typically comprises three coordinates: aradial coordinate, an angular coordinate, and an elevation (r, θ, and z,respectively). A cylindrical coordinate system may be desirable where,for example, all Wi-Fi emitters have the same elevation. Angles may bedetermined as described above.

In some embodiments of the present invention, sound waves may be used toperform one or more of: location determination, movement tracking ininterior or exterior locations, and distance calculation from a positionto an Employee, which may be accomplished based upon transmission andreceipt of sonic transmission. Sound wave transmissions include severalsignificant attributes, which may be beneficial for a given set ofcircumstances when used for radiofrequency-based location determination.According to the present invention, sonic waves may be deployedindependently, or in combination with, transmissions and reception oflogical communications utilizing other bandwidths, such as bandwidthsassociated with Wi-Fi, Bluetooth, ANT, infrared or almost any wavelengthin the Industrial, Scientific and Medical bands (sometimes referred toas “ISM Bands”). Methods, devices, and apparatus for using sound wavesin location determination may be found, for example, in U.S. Pat. No.10,628,617, the contents of which are incorporated herein.

In certain embodiments of the invention, a direction of interest of anAgent or Smart Device may be determined. In some cases, a directiondetermination may be based upon a movement of a device. For example, adevice with a controller and an accelerometer, such as mobile SmartDevice, may include a user display that allows a direction to beindicated by movement of the device from a determined location acting asa base position towards an As Built feature in an extended position. Insome implementations, the Smart Device may first determine a firstposition based upon triangulation with the reference points and a secondposition (extended position) also based upon triangulation with thereference points. These position determinations may proceed as describedabove. The process of determination of a position based upontriangulation with the reference points may be accomplished, for examplevia executable software interacting with the controller in the SmartDevice, such as, for example via running an app on the Smart Device.

FIG. FIG. Referring now to FIG. 2 , according to the present inventionan Agent, such as a human person 201, supports one or more Transceivers204-212 within a wireless communication area, such as an interior of astructure 200. In some examples, the structure may be a cold storagefacility or an apparatus such as a cold storage truck that may havestructure to it. The Agent supported Transceivers 204-212 may beoperative to wirelessly communicate with Reference Point Transceivers202A-D located within the structure 200. Information included inwireless communications between the Reference Point Transceivers and theAgent supported Transceivers 204-212 include values for variables thatmay be used to generate a position of the Agent supported Transceivers204-212.

The Agent supported Transceivers 204-212 are co-located with the Agentresulting from being supported by the Agent, therefore a position of theperson 201 may be designated as being the same as the position of one ormore of the Agent supported Transceivers 204-212 or sonic mathematicalcorrelation of the respective positions of the Agent supportedTransceivers 204-212, such as for example: an average, a weightedaverage, a mean, a median, other function or algorithm involving therespective positions of two or more to the Agent supported Transceivers204-212.

In some embodiments, a person 201 (or other Agent or mammal) willsupport two or more Transceivers 204-212 and each of the Agent supportedTransceivers 204-212 will enter into communication with the ReferencePoint Transceivers 202A-D in a manner conducive to generating arespective position of the two or more Agent supported Transceivers204-212 supported by the person 201. A directional vector and/or ray maybe calculated based upon respective positions of two or more of theAgent supported Transceivers 204-212 supported by the person 201. Thedirectional vector and/or ray may be used, for example, to designate aforward facing position of the person 201 or a direction of interestassociated with the person 201. For example (and discussed in moredetail below) headgear 251 may include multiple headgear mountedtransceivers 211, such as a transceiver 211 along a front portion of aheadgear (front brim or front headband portion or from of eyeglasses)and a second headgear transceiver along a rear portion of the headgear(not shown in FIG. 2 ) a directional vector and/or ray may be generatedfrom the position of the transceiver along a rear portion of theheadgear through a front portion of the headgear (defined by theposition of the front portion headgear transceiver 211) and a frontwardfacing direction (and/or a rearward or sideways facing direction) may bedesignated based upon the directional vector/ray. The directionalvector/ray may be used to determine if the agent was facing in adirection of a item stored in the cold storage.

Other wearable items 252-257 or subcutaneously implanted items, whichmay also be considered a wearable item 258, may also includeTransceivers 204-212, such as those supported at areas of the person 201other than the person's head. By way of example, Transceivers 204-212may be attached to, incorporated into, mourned on, held by, or otherwisesupported by items wearable by a person 201 or items that may be held inthe person's hand.

A subcutaneously implanted item, such as wearable item 258 may includesensors of various kinds to measure physiological measurements as wellas communication and identification devices. A subcutaneously implanteditem ma have art outer casing consistent with insertion under the skinsuch as, in a non-limiting sense, glass or hydrogels. The implant maymeasure temperatures, pressures, electrical signals and the like. Insonic examples, the subcutaneously implanted item may include antennasfor the reception of signals and energy. The subcutaneously implanteditem may be paired with other devices such as transponders and sensorsto provide sensed measurements, location and orientation information.For purposes of discussion, some examples of subcutaneously implantedsensors and devices may be classified as a type of wearable item in thediscussion herein.

The present invention additionally provides for Sensors 204-212 (withadditional sensors illustrated and discussed in subsequent figures) thatare supported by the person 201 or located within the structure 200. Thesensors quantify a condition within the structure 200. For example, thecondition may be a physical state present in the structure or aphysiological state present in the person 201.

In some preferred embodiments, Sensors may be placed on an inner surfaceof a wearable item such the sensor has access to a skin surface on theperson 201. Access to a skin surface may allow, or improvequantification of a physiological condition (sometimes referred toherein as a Biometric). For example, access to a skin surface may allowfor electronic quantification of a body temperature, a heart rate, ablood oxygen level, a blood sugar level, blood pressure, intracranialpressure and other conditions present and quantifiable via electronicsensors supported by the person 201.

In other embodiments, a structure sensor 213 may be positioned, such asmounted on an architectural aspect or equipment item, and receive energyfrom an environment around the sensor in a manner that allows thestructure sensor 213 to quantify a condition within the environment. Forexample, a structure sensor 213 may receive environmental input 241 suchas infrared energy into the sensor and based upon the receipt ofenvironmental input 241, such as for example infrared energy, thestructural sensor 213 may quantify surface temperatures of items withinthe structure 200. The items within the structure may include person(2)and the structural sensor may quantify a respective surface temperatureof each person 201-201A. if a structural sensor 213 is positioned toreceive environmental input, such as infrared energy 241 from a knownarea, the position of a person 201-201A may be determined via wirelesscommunications (as discussed herein) and a quantified temperature may becorrelated to a particular person based upon the location of the person201-201A and the known area monitored and quantified by the structuralsensor 213.

In another aspect, a location of Agent supported Transceivers 204-212may be determined via wireless communications with Reference PointTransceivers 202A-D and used to designate a forward facing direction ofa person 201 at a particular instance in time. Therefore, one or moreof: triangulation, trilateration and determination of an angle oftransceiving may be used to designate a forward facing position. Forexample, an AoD and/or AoA may be used to designate a frontward facingdirection (or rearward, sideways facing direction) at a particularinstance in time (and/or timeframe) while triangulation is used togenerate a geospatial position during the same timeframe (or anoverlapping timeframe).

As illustrated in FIG. 2 , as well as in FIGS. 2A-2F, according to someembodiments of the present invention, sensors 214-236 are also placed inand/or on a wearable item 251-258. The sensors may quantify a conditionin an environment that is ambient to the person 201 or quantify acondition experienced by the person 201. For example, a sensor worn in awrist strap 253 type item, such as a watch or a compressive band orother type wrist strap 253 may include one or more sensor(s) thatquantify a condition experienced by the person 201 by measuringphysiological conditions experienced by the person, such as, forexample: a body temperature, a heart rate, a skin temperature, shaking,falling, impact, vibration, acceleration, deceleration, remainingstationary, appendage movement, head movement, eye movement, and thelike.

Sensors in other wearable items, may quantify similar or variantconditions. For example, an accelerometer or piezo device in a footwear256 may quantify steps. A Transceiver in a ring 212 may quantify handmovement, such as a natural swing movement during walking or running, arotational movement while operating a steering wheel, a verticalmovement while lifting or climbing a ladder, etc., and in someembodiments may work in correlation with an image capture device, suchas stereoscopic cameras in a head gear 251 to quantify hand and fingermovements that may be processed and translated into control commands.

An eye covering 257 type item supported by the person may include, forexample, eye glasses, goggles, an augmented reality (A/R) headset, avirtual reality (V/R) headset, facemask or other item that is generallyplaced in position in front of a person's 201 eyes may include one ormultiple transceivers 205 that may transceive to help determine aposition of the person 201 wearing the eye covering 257. The one ormultiple transceivers 205 may also transceive in a manner that allowsfor determination of a forward direction of the eye covering 257 suchthat a designation of a direction the person 201 is facing may becorrelated with the forward direction of the eye covering 257 Theforward direction of the eye covering may be determined according to themethods and devices discussed herein for determining a direction, suchas a direction of interest.

In another aspect, one or more sensors 237 mounted on the eye covering257 may monitor and quantify eye movement and a direction of eye focus.A direction of eye focus quantified by the sensor 237 may assist indetermining one or more of: a forward facing direction, what the eye isfocused on, whether the person 201 is looking down or up, nystagmusmovement or her health and/or performance condition.

As discussed above, Transceivers 212-236 may be mounted on a wearableitem 251-258 such that as a wearable items with one or more Transceivers212-236 and a Transceiver 204-212 may quantify a condition experiencedby the person 201 wearing the wearable item 251-258. A conditionexperienced by the person may include one or more physiological state(s)of the person 201, such as, for example, a heartrate, a bodytemperature, a breathing rate, breathing pause, eye movement, bodyconductivity, body capacitance, or other biophysical and/or biochemicalstate or condition.

Sensors incorporated into a wearable item may therefore providequantification of bodily conditions present in the wearer. Thequantified body conditions may be transmitted via a wirelesscommunication to a processor that is operative to identify a bodilycondition. For example, the processor may check to ascertain if aquantified body condition is within thresholds determined to indicate ahealthy condition of the body; or if the quantified body conditionindicates a transitory state that may ultimately result in an unhealthystate; or if the quantified body condition indicates a present state ofa human body that is in an unhealthy state.

As further discussed herein, a condition quantified by a sensor mayprecede execution of a remedial action based upon a value of a variableused to represent the condition quantified by the sensor. In thoseembodiments that include a body condition being quantified by a sensorworn in a wearable item 251-258 by a person 201, a remedial action mayinclude, by way of non-limiting, one or more of: alerting the person 201wearing the wearable item 251-258. An alert to the person 201 wearingthe wearable item 251-258 may include, for example, a human audiblealert, a kinetic action, such as a vibration or TENS (Transcutaneouselectrical nerve stimulation), a visual indication (such as a light), oractivation of another device perceptible by the person 201, An alert mayalso be generated that is communicated to a person other than the person201 wearing the wearable item 251-258, or a server, processor,controller or other automation. For example, a supervisor in aprocessing plant may receive one or both of an email and a text messageindicating that a person 201 wearing a wearable item 251-258 and workingin the processing plant has a body temperature that exceeds 100° F. andan elevated heartrate. The supervisor may recognize that a fever andelevated heartrate may be indicative of a viral infection and thereforecontact the person 201 wearing the wearable item 251-258 and direct theperson 201 to be further assessed. In addition, because the person willalso be associated with wireless position tracking via a. transceiverincluded in a wearable wrist band, the present invention may determinewhere the person 201 has moved about while the person 201 has generatedan indication of a fever, e.g. an elevated body temperature, Moreover, aposition of a first person 201 may be determined relative to a positionof a second person 201A during one or more time intervals. Additionalremedial actions may be implemented if the first person 201 has beenlocated at a position within a threshold distance 240 from the secondperson 201A. For example, both the first person 201 and the secondperson 201A may be placed in quarantine until such time as it isdetermined that both the first person 201 and the second person 201A arefree of viral infection. Alternatively, in some embodiments, acomparison of a first pattern of wireless positions indicative of a pathtravelled by a first person 201 and a second pattern of wirelesspositions indicative of a path travelled by a second person 201A mayindicate that the second person 201A is not likely to contract a virusfrom the first person 201 because the second person 201A did not comewithin a distance 240 determined to be statistically favorable totransfer of the virus.

In some embodiments, Transceivers 212-236 may quantify almost anycondition present in an area defined by a range of a Reference PointTransceiver 202A-202D communicating with a Transceivers 204-212associated with a Transceivers 212-236. Embodiments will include an areaof communication between the Transceiver 204-212 and the Reference PointTransceivers 202A-202D that includes an interior of a facility or otherstructure 200, and an area immediately surrounding the structure. Asdiscussed above, a condition quantified by sensors within range of theReference Point Transceivers 202A-202D may be a physiological state of aperson 201. In addition, the condition quantified by sensors withinrange of the Reference Point Transceivers 202A-202D may include acondition in an environment within the area, such as an atmosphericenvironment and/or a condition in a structure 200 or an item ofmachinery.

As such, in some embodiments, a wearable item 251-258 that includes awristband, headgear, vest, gown, footwear, or other capable of assessinga condition of a person 201 and an environment item, may include sensors214-236 that quantify a condition present in the person 201 that wearsthe wearable item 251-258 or a condition ambient to the person 201wearing the wearable, item 251-258. Non-limiting examples of conditionsambient to a person include, temperature, humidity, visible light, IRlight, UV light, other electromagnetic energy within a definedbandwidth, movement, vibration, impact, noise, motion or otherquantifiable condition.

In some embodiments, one or more sensors 213 may be positioned otherthan as part of a wearable item 251-258. For example, a sensor 213 maybe fixedly attached to a structure 200. The structural sensor 213attached to the structure 200 may receive environmental input 241descriptive of an environment surrounding a person 201-201A. The presentinvention is able to generate a position of a person 201-201A, quantifya condition in an environment 238 surrounding the person 201-201A,quantify a condition present as a physiological state of the person201-201A at multiple instances in time and store each position andquantified condition. In addition, the present invention may generate auser interface indicating a present status of the above listedconditions, a chronology of states of conditions and positions, asummary (including mathematical analysis, such as average, mean,weighted average, median etc.).

The present invention may also provide a user interface or other humandiscernable device, such as a light, audio Instruction, mechanicalactuation, video, narrative or other communication medium to indicatewhat actions the controller deems to be appropriate based upon aparticular set of conditions quantified by the sensors 214-236 andalgorithmic analysis, such as unstructured queries and artificialintelligence. For example, for a viral contagion, such as, for example,COVID-19, a close contact may be defined as anyone or any tracked itemthat Nus within six feet of an infected person for at least 15 minutesstarting from 48 hours before the person began feeling sick, if it isknown that the virus propagates well in an environment with certaintemperature and humidity conditions and does not propagate well in anenvironment with other temperature and humidity conditions, then thepresent invention may provide an interface indicating a first value forremedial action, such as a distance 240 between persons 201-201A tomaintain, or what should be considered “close contact” based upon afirst set of environmental conditions that are quantified, and a secondvalue for that should be considered “close contact” for a second set ofenvironmental conditions.

More specifically, if an exemplary virus propagates well in anenvironment of less than about 10° C. and in humidity of less than about75%; and the same virus has limited propagation in environments of over20° C. and atmospheric humidity of about 80% or more, then the presentinvention may provide an interface or other indicator of a distance 240defining a first value for a “close contact” based upon a temperatureand humidity of an environment in which a people are situated; e.g. afirst value defining a distance 240 between persons 201-201A for closecontact as 6 feet or less and a second value defining a distance 240between persons 201-201A for close contact as 12 feet or less. A choiceof the first value for dos contact and the second value for closecontact will be contingent upon measured quantifications of atemperature and humidity of an environment 238 in which a person201-201A is situated. Other quantified environmental conditions maybecome a factor upon which user interface values are based.

In addition to a suggested distance 240 to maintain between persons201-201A, the present invention provides for sequential positiondetermination for persons 201-201A based upon wireless communication(position based upon wireless communication is discussed in more detailherein). According to the present invention, wireless communicationincluding values of variables capable of determining a position of atransceiver supported by an Agent, such as a person 201-201A isconducted on a periodic and/or episodic basis. Periodic basis includes atimes period between wireless communications, such as every t seconds,or every minute, or every five minutes or other time period appropriatetoile circumstances, such as a likelihood of movement and powerconsumption. A wireless communication based upon an episodic basis mayinclude wireless communicating as a response to an event, such asmovement detected by a tower powered device, such as accelerometers,solid state magnetic sensors, gimbals and the like, or other relativelylow powered device that is capable of detecting movement. Other eventsmay also precede a wireless communication, such as a detection ofanother person within a defined distance 240, a change in anothervariable, such as temperature, carbon dioxide level, inert gas level,oxygen level, presence of a chemical, or almost any other measurablequantity. Essentially, if a threshold level of a quantifiable conditionis reached, an onboard controller is programmed to conduct wirelesscommunication and calculation of a position of a transceiver conductingthe wireless communication.

In another aspect, environmental conditions and person conditions may bemeasured by sensors included in wearable items worn by a person, or bysensors located in a position conducive to quantifying a condition ofone or both of an environment 238 in which the person 201-201A islocated and a position conducive to quantifying a condition inherent inthe person 201-201A. For example, a sensor 213, such as an infrared(IR), thermistor, thermocouple, or other sensor type may receiveenvironmental input 241 that the sensor 213 may use to quantify acondition, such as, a temperature of the environment 238 or a conditionof a person 201-201A, The sensor 214-236 may be located in a wearableitem 251-258, or a sensor 213 may be fixed or removably fixed, in aposition in a structure 200. Either way a sensor 213-236 may beoperative to quantify a temperature condition of a person 201-201A or anenvironment 240. A value generated by a sensor 213-236 may be stored andor processed by a controller. Processing may include a mathematicalcombination of a sensor reading with other sensor readings. A sensorreading may include a sensor generated value of a variable. Combining ofvalues of variables may include aggregating and mathematicallyprocessing the sensor generated values of variables.

As illustrated, a wearable item 251-258 may include, by way ofnon-limiting example, headgear 251, such as a hat, hardhat, headband,skull cap, cap, A/R headset, V/R headset, surgical cap, scrub cap; anearring 252; an wrist strap 253, including an arm band, a watch, abracelet, or other item that may be positioned around an arm; a torsocovering 254, such as a vest, lab coat, scrub; hazmat suit, smock; aring 255, footwear 256, such as a shoe, boot, sandal, slipper, cleat,sneaker, or other article fitted to a foot. As illustrated, each of thewearable items illustrated 251-258 has a Transceivers 204-212 associatedwith it. The Transceivers 204-212 may be fixedly attached, removablyattached or incorporated into the wearable item.

In some embodiments, a wearable item 251-258, such as for example a ring255, an earring 252, or any of the other wearable items may include atransceiver that engages in low power, short distance communication,such as for example, one or more of: LPWAN, WhiskerRF, ANT, Near FieldCommunications (NFC), Zigby, Bluetooth Low Energy, or other low energytype communication. The wearable item may then communicate sensor datato a higher powered smart device 239, such as one or more of: acommunications Tag, a Node, a Smart Phone, a Smart headset, a smartwatch and the like, wherein the higher powered smart device 239 mayengage in wireless communication with one or more Reference PointTransceivers 202A-202D to determine a position of a person 201-201A andalso convey through the wireless communications some or all of thedigital content generated by sensors 214-236 included with the wearableitem 251-258.

For example, in some embodiments a higher powered smart device 239 mayreceive from the lower powered sensor multiple values of a variablequantifying a condition present in a structure 200, each value of thevariable quantifying a condition present in the structure associatedwith a chronological time value. Variables, and values for variableswill depend upon a type of sensor made operative to quantify acondition. Sensors may be chosen based upon which conditions will bequantified. Similarly, a transceiver 202A-213 may vary based upon anenvironment in which the transceiver will be deployed.

Sensors 214-236 may include an accelerometer to quantify motion(movement, acceleration); magnetic sensor, which may include magneticfield sensors, Hall Effect Sensors and/or Anisotropic Magneto-Resistive(AMR) sensors, magnetometer (3-axis); ambient temperature sensor;ambient light sensor; specific wavelength of light sensor; viii) ambienthumidity; Magnet detector (maintenance button, “docked” indication,etc.); ix) sound pressure level (SPL); optical proximity sensor; powersource such as a rechargeable lithium polymer battery; USB connector forcharging and/or data transfer; NFC (near field communications) (tag, orreader); Battery voltage sensing; audio alert, such as a Beeper;Vibrator, LED indicator(s); Manual switch such as a tactile button;Inductive charging; vibration detection; air quality sensing such asCO2, VOC, and the like; specific gas detection (CO, CO2, H2S,particulate); Gesture sensing (sense hand swipes over device); tap anddouble tap detection; Agent fail detection; free-fail detection; shockand/or impact detection; IR (infrared) beacon TX and/or RX; pedometer;activity meter; processor enhanced audio monitoring (e.g. listen tierfire alarm siren); heartrate monitor; blood oxygen sensing; OLEDdisplay; electronic ink display; and most any other electronic sensor ofa quantifiable condition.

By way of non-limiting example Transceivers 202-212 may be operative toperform wireless communications suitable for real time locationfunctionality and/or data transfer, transmitting and receivingcapabilities using one or more modalities such as: UWB; Bluetooth,including BLE; WiFi RTT; infrared; ultrasonic; sonic; sub GHz; or othermodality.

In some embodiments, communication between transceiver, such as thoseincluded in Tags, Nodes and/or Smart Devices may be achieved by a bursttransmission protocol, such as low-power derivative of Gaussian pulses.In exemplary embodiments, the burst transmission protocol may transmitradio waves having a frequency of 3.0 GHz-11.0 GHz, with a highbandwidth of approximately 500 MHz-1.3 GHz, in some embodiments, thebandwidth may be based on a tolerance from an arithmetic centralfrequency; for example, if the central frequency for the bursttransmissions is 6.0 GHz, then the bandwidth may be a relativetolerance, such as 20% of 6.0 GHz (i.e., 1.2 GHz). An interval betweenindividual pulses may be uniform or variable, based on an implementedencoding scheme. In some exemplary embodiments, burst transmissions maylack carrier waves. Transceiving modalities that may be implementedusing burst transmission protocol according to the present invention,include, ultra wideband, sub-GHz, and industrial, scientific and medical(“ISM”) radio bands within the frequency range of 6.7 MHz (megahertz)and 250 GHz.

In sonic exemplary embodiments, one or more pulses of wirelesscommunication may be relatively short in duration (i.e., 0.01 ns-10 ns).The total pulse train may be written as the sum of individual,time-shifted pulses; i.e., s(t)=Σ_(n=0){circumflex over ( )}∞

a_np(t−τ_n)

where s(t) is the pulse train signal, p(t) is a characteristic pulseshape, an is the amplitude of the nth pulse, and τn is the time-offsetof the nth pulse.

When two nodes exchange burst transmissions, they may be able to detecta range or distance between the two nodes. This may be accomplishedthrough a “time of flight” (ToF) measurement. The ToF measurement may bemade by or supplemented with a time difference of arrival (TDOA) or timedelay estimation (TDE). TDOA may rely on a measurement of the amount oftime it takes for a signal transmitted by one node to reach a pluralityof other nodes. Using multilateration techniques, a location of thetransmitting node may be estimated.

Referring now to FIGS. 2A-2E, in various embodiments, a wearable item251-256 may take appropriate forms adapted to be worn or otherwisesupported by various parts of a human person 201-201A. For example, FIG.2A illustrates an arm supported wearable item 253 which may be in theform of a wrist band or arm band or other item that is securable orotherwise supported on an arm of a human person. FIG. 2A alsoillustrates a finger supported wearable item 255 in the form a ring orother device securable upon a finger of a human person 201-201A. FIGS.2B-2C illustrate torso supported wearable items 254-254A including avest 254 and a lab coat 254A or other gown type apparel. FIG. 2Dillustrates footwear 256 and FIG. 2E illustrates a wearable item in theform of a headgear 251.

Each wearable item 251-256 may include at least one wireless transceiver204-212 and one or more sensors 213-236. As discussed herein, thewireless transceiver 204-212 may include a solid state componentsoperable to transmit and/or receive wireless communications. Thewireless communications may utilize any of the modalities discussedherein. The transceivers may wirelessly communicate with one or moreReference Point Transceivers 202A-202D and/or a smart device 239, suchas a smart phone, smart tablet, smart headgear and the like. Wirelesscommunications may include relatively higher powered and longer rangemodality, such as UWB, Bluetooth, WiFi, cellular or satellitecommunications and/or lower powered and shorter range communicationmodality, such as Zigby, ANT, BLE, or other low power protocol.

Those wearable items 251-256 that are capable of supporting more thanone wireless transceiver may be operative to engage in wirelesscommunications with Reference Point Transceivers 202A-202D and generatepositional coordinates for each transceiver 204-212 (as illustratedwearable items 251, 254, 256 include multiple transceivers, but anywearable item with a large enough area may include multiple transceivers204-212). Two or more sets of positional coordinates may be referencedto generate a direction of interest. In the present invention adirection of interest may include a direction a person 201-201A isfacing. Accordingly, a headgear 251, a vest 254 and/or a shoe 256 mayinclude multiple transceivers 211A-211D, 207-209 and 204-204Arespectively for which corresponding positional coordinates may begenerated based upon values of variables transceived according to themethods an apparatus described herein. The multiple positionalcoordinates may be used in turn to generate a direction of interest,such as, a forward facing direction. A forward facing direction may beimportant in certain safety situations, such as in the case of acontagion that may be spread by a cough, sneeze of exhale. It may beimportant to know a separation distance 203 between a first person 201to a second person 201A in the forward direction. A separation distance203 in a rearward direction may not have as short a critical distance.

In some embodiments, Sensors 213-236 are hardwired or otherwise inphysical logical communication with connected to transceivers 204-212included in a same wearable device. In physical logical communicationmay include, by way of non-limiting example, on a same printed circuitboard (PCB); in a same semiconductor chip; connected by a circuit traceror wired connection; or other physical logic pathway capable ofconveying a digital value.

Transceivers 213-236 may be operative to engage in wirelesscommunication with one or more of: Reference Point Transceivers202A-202D; a smart device 239, such as a Smart Phone, Smart Tablet orSmart Headgear; a cellular network; a satellite network; a mesh networkor other logical network.

Referring now to FIG. 2F a first person 201 and a second person 201A areillustrated in place to interact with processing equipment 260.Transceivers 207-208 may be supported by the persons 201-201A and engagein wireless communications 243 with one or more Reference PointTransceivers 202A-202D, and values for variables included in thewireless communications 243 may be used to generate location coordinatesindicating a respective location 263-264 for the persons 201-201A.Respective locations 263-264 may be referenced to determine separationdistance 203 between persons 201-201A and also a separation distancebetween a person 201A and an item of equipment 260 or other itemsurface, such as a surface of a product 261. For products such as fooditems or pharmaceuticals, contamination with a contagion may beparticularly sensitive. Separation distance 203-203A may be used todetermine a risk on contamination with a contagion. A length of time mayalso be determined based upon a sequence of wireless communications 243at known time intervals, wherein the wireless communications are used togenerate a location 263-264 of an Agent, such as a person 201-201A.

Ongoing determination of a location 263-264 of an Agent, such as aperson 201-201A may also be used to ensure that the person 201-201 Adoes not traverse and area designated as a contaminated area, such as araw food area, or a chemically contaminated area, and then proceed to acontrolled area, such as a non-contaminated area, such as an area withfood or other product ready for distribution to the public.

Devices involved in the generation and/or storage and/or transmitting ofdigital content, such as, one or more of: agent supported transceivers207-208, reference point transceivers 202A-202D, gateways 270, andservers 271 may each engage in direct logical communications and/or inlogical communications with an intermediary device For example, areference point transceiver 202A-202D may engage in logicalcommunication with a gateway 270 or other intermediary device and thegateway 270 may engage in logical communication 269 with a server 271;or the reference point transceiver 202A-202D may engage in logicalcommunication 267 directly with the server 271. There may be numerouspaths of logical communication 265-269 between various device types. Invarious embodiments of the present invention, any or all of the devices(e.g. sensors, transceivers, gateways, servers) involved in generationand/or storage and/or transmitting of digital content, may aggregate,store and or process digital content that the device has access to.

Also, variations on a location of a sensor and a transceiver are withinthe scope of the present invention. As discussed above, a Sensor 204-213may be located on a wearable item 251-258 or a sensor 213 may bepositioned at a point extraneous to a person 201-201A, such as mountedon an architectural aspect (such as, for example, a doorway, a hallentrance, a turnstile or other crowd movement control device; andreceive sensor input 242 from a person 201 useful to quantify acondition of the person 201. Transceivers 207-208 may determine alocation of a person 201-201A that correlates with a sensor 213quantification of a condition present with the person 201-201A basedupon input 242 received from the person 201-201A. Based upon a location263 of a person 201, a processor (such as those present in a gateway270, server 271 or smart device 239) which person is associated with acondition quantified by the sensor 213 mounted on an architecturalaspect.

Some embodiments may also include a sensor 213 capable of receivinginput 242 sufficient to quantify biometric based identification of theperson 201, and/or a physiological state present in the person 201. Forexample, sensor 213 may receive input enabling facial recognition of theperson 201 and correlate input from a sensor 204-213 to quantify aphysiological condition and a facial recognition identification.Transceivers 207-208 may engage in wireless communications sufficient todetermine a location 263 of the person 201 when the physiologicalcondition and a facial recognition identification took place via othersensors than those co-located with the transceivers supported by theAgent.

Some modalities, such as those modalities that adhere to the Bluetooth5.1 or BL5.1 standards, allow a Node to determine an angle of arrival(AoA) or an angle of departure (AoD) for a wireless transmission. Anarray of antennas may be used to measure aspects of the Bluetoothsignaling that may be useful to calculate these AoA and AoD parameters.By calibrating an antenna system, the system may be used to determineangles in one or two dimensions depending on the design of the antenna.The result may be significant improvement in pinpointing the location oforigin of to signal.

An array of antennas may be positioned relative to each other and atransmitting transceiver to allow for extraction of an AoA/AoD. Such anas may include a rectangular array; a polar or circular array; a lineararray; and a patterned array, where a number of antennas are deployed ina pattern conducive to a particular environment for transceiving.Antennas may be separated by characterized distances from each other,and in some examples, a training protocol for the antenna array resultsin antenna positioning incorporating superior angle and locationprecision, Some transceivers may transceive in 2.4-2.482 GHz frequencybands, and thus the radiofrequency transmissions may have wavelengths inthe roughly 125 mm as length scale. A collection of antennas separatedby significantly less than the wavelength may function by comparing aphase of RF transmissions arriving at the antennas. An accurateextraction of phase differences can yield a difference in path lengththat when accumulated can lead to a solution for the angles involved. Insome embodiments, Nodes may include antenna arrays combined withbatteries and circuitry to form complete self-contained devices. Antennaarrays and methods of using the same for determining position anddirection of a Smart Device or other Node are described in U.S. Ser. No.16/775,223, the contents of which are incorporated herein by reference.

Referring now to FIG. 3A, an example according to the present inventionillustrates an Agent, such as a human person 201, supporting one or moreTransceivers 204-212 within a wireless communication area, such as aninterior of a structure 200. Many of the elements are similar to theelements of FIG. 2 and the referenced elements with numerals in the200's having the same meaning. However, instead of using transceiversand sensors to characterize interactions between multiple agents, theexample of FIG. 3A uses transceivers and sensors to characterizeinteractions between agents and physical items. For example, thephysical item 300 may reside in a cold storage location. The physicalitem 300 may support one or more Transceivers 311-314 and one or moresensors 320. In manners described in various examples and embodimentsherein, the transceivers 311-314 may be used to characterize theposition and orientation of the physical item 300. The positions ofthree Transceivers of the example may be used to determine a planerepresenting the face of the physical item 300 and its location. Thus,communications may be used to store data from the transceivers andsensors of both the agent 201 and the physical item 300 which may beused to characterize the interaction between them such as in anon-limiting example the distance 303 separating them at given timepoints as well has whether they faced each other at the respective timepoints.

The recording of data representing the interactions between agents anditems in cold storage could have numerous important functions. Forexample, the various sensors on the agent could store data relevant tothe health status of the agent. The health status indicators can becoupled to the position and orientation information to indicate if it islikely that the agent came sufficiently close to an item in cold storageto effect it is some manners.

The physical item 300 in a cold storage location may also record ambientconditions such as temperature, particulate level and air pressures.Variations in the measurement of temperature may form a record thatcould be used to disposition the integrity of the physical item kept incold storage. The sensor data may also be useful in determininglikelihood of contamination events between the agent and the physicalitem. More generally, the sensors on the physical item 300 may recordambient condition data and communicate that with the controlling systemsof the cold storage location. FIG. 3A represents interactions betweenagents and physical items, however, the sensing systems and thecommunications systems of the structure may interact with the physicalitem in the absence of any agents and may be used to record ambientconditions, variations and the like. The systems may also be used toadjust and regulate conditions within the cold storage structure.

Retelling now to FIG. 3B, methods and devices for determining adirection that may be referenced for one or both of data capture anddata presentation of a particular portion of the virtual representationof surroundings of a user. An Agent 350 may position a Transceiver 355in a first position 351 proximate to a portion in a space of interest375. The first position 351 of the Transceiver 355 may be determined andrecorded. The Agent 350 may then relocate the Transceiver 355 to asecond position 352 in a general direction of the portion in space ofinterest. With associated position information obtained for the firstand second positions a controller in an external system or in anassociated Transceiver 355 may generate one or both of a ray and avector towards the portion of a space of interest.

In some embodiments, the vector may have a length determined by acontroller that is based upon a distance to a feature of interest inspace as represented in a model on the controller in the direction ofthe generated vector. Tile vector may represent a distance 353 from thesecond position 352 to the space of interest 375 along the axis definedby a line between the first position 351 and the second position 352. Incontrast, a ray may include just a starting point and a direction.

In still other embodiments, a device with a controller and anaccelerometer, such as mobile phone, tablet or other Smart Device thatincludes a Transceiver 355, may include a user display that allows adirection to be indicated by movement of the device from a determinedlocation acting as a base position towards an point in a direction ofinterest or representing a center of an RTA of the device. The movementmay occur to a second location in an extended position. In someimplementations, the Smart Device determines a first position 351 basedupon triangulation with the reference points. The process ofdetermination of a position based upon triangulation with the referencepoints may be accomplished, for example via executable softwareinteracting with a controller in the Smart Device, such as, for exampleby running an app on the Transceiver 355.

An array of antennas positioned at a user reference point may allow forthe accurate receipt of orientation information from a transmitter. Asdiscussed earlier a combination devices with arrays of antennas may beused to calculation a position. A single Node with an array of antennascan be used for orienteering and determining a direction of interest.Each of the antennas in such an array receiving a signal from a sourcemay have different phase aspects of the received signal at the antennasdue to different distances that the emitted signal passes through. Thephase differences can be turned into a computed angle that the sourcemakes with the antenna array.

Referring to FIGS. 4A-D a series of exemplary devices employing matrices(or arrays) of antennas for use with Nodes that communicate wirelessly,such as via exemplary UWB, Sonic, Bluetooth, a Wi-Fi or other modality,is illustrated. Linear antenna arrays 401 are illustrated in FIG. 4A.Rectangular antenna arrays 402 are illustrated in FIG. 4B. Circularantenna arrays 403 are illustrated in FIG. 4C, other shapes for arraysare within the scope of the invention. In addition, an antenna array maybe omni-directional or directional.

In some embodiments, see FIG. 4D item 404, a Smart Device 405 mayinclude one or more Nodes 406 internal to the Smart Device 405 orfixedly attached or removably attached to the Smart Device 405. EachNode 406 may include antenna arrays combined with a power source andcircuitry to form complete self-contained devices. The Nodes 406 or acontroller may determine an RTT, time of arrival AoA and/or AoD or otherrelated angular determinations based upon values for variables involvedin wireless communications. For example, a composite device 404 may beformed when a Node 406 with a configuration of antennas, such as theillustrated exemplary circular configuration of antennas 407, isattached to a Smart Device 405. The Node 406 attached to the SmartDevice 405 may communicate information from and to the Smart Device 405including calculated results received from or about another Node 406,such as a Node 406 fixed as a Reference Point Transceiver or a Node withdynamic locations, wherein the wireless communications are conducive togeneration of data useful for determination of a position (e.g. timingdata, angles of departure and/or arrival, amplitude, strength, phasechange, etc.). A combination of angles from multiple fixed referencepoints to the antenna array can allow for a location of a user in space.However, with even a single wireless source able to communicate with theantenna array, it may be possible to determine a direction of interestor a device related field of view.

An array of antennas positioned at a reference point may allow for theaccurate receipt of orientation information from a transmitter. Asdiscussed earlier a combination devices with arrays of antennas may beused to calculation a position. A single Node with an array of antennascan be used for orienteering and determining a direction of interest.Each of the antennas in such an array receiving a signal from a sourcewill have different phase aspects of the received signal at the antennasdue to different distances that the emitted signal passes through. Thephase differences can be turned into a computed angle that the sourcemakes with the antenna array.

Referring now to FIG. 5 , in some embodiments, one or both of a SmartDevice 501 and a smart receptacle 502 housing Smart Device 501 may berotated in a manner (such as, for example in a clockwise orcounterclockwise movement 520-522 relative to a display screen) thatrepositions one or more wireless position devices 503-510 from a firstposition to a second position and moves a top edge 518 and a bottom edge519 of the Smart Device 501. Multiple wireless Reference PositionTransceivers 511-514 may support position determination. A vector 526may be generated at an angle that is perpendicular 525 or some otherdesignated angle in relation to the Smart Device 501. In someembodiments, an angle in relation to the Smart Device is perpendicular525 and thereby viewable via a forward-looking camera on the SmartDevice. An Agent may position the Smart Device 501 such that an objectin a direction of interest is within in the camera view. The SmartDevice may then be moved to reposition one or more of the wirelessposition devices 503-510 from a first position to a second position andthereby capture the direction of interest via a generation of a vectorin the direction of interest.

Referring now to FIG. 6 , as illustrated, a vector 625 indicative of adirection of interest 625 may be based upon a rocking motion 623-624 ofthe Smart Device 601, such as a movement of an upper edge 618 in aforward arcuate movement 623. The lower edge 619 may also be moved in acomplementary arcuate movement 624 or remain stationary. The movement ofone or both the upper edge 618-619 also results in movement of one ormore wireless position devices 603-610. The movement of the wirelessposition devices 603-610 will be a sufficient distance to register togeospatial positions based upon wireless transmissions. A requireddistance will be contingent upon a type of wireless transmissionreferenced to calculate the movement. For example, an infrared beam mayrequire less distance than a Wi-Fi signal, and a Wi-Fi transmission mayrequire less distance than a cell tower transmission which in turn mayrequire less distance than a GPS signal. In some embodiments, asdiscussed further above, hybrid triangulation may include one or moredistances based upon wireless transmissions of different bandwidths ormodalities. For example, a first modality may include Wi-Fitransmissions and a second modality may include Bluetooth transmissions,still another modality may include infrared or ultrasonic modalities.

Referring to FIG. 7 , line segments 731-738 are illustrated thatintersect various generated position points (PP1-PP8) for Transceivers703-710. A Smart Device 701 is illustrated with the array or collectionof transceivers 703-710 arrayed along the edges 702 of the device.Position points PP1-PP8 may be generated according to the methods andapparatus presented herein, including a mathematical average, median, orother calculation of multiple positions determined via triangulationtechniques. In addition, a vector 739 or ray may be generated based uponone or more of the lines 731-738. Wireless position transponders 711-713may communicate with various devices. In some embodiments, positionpoints may be recorded and sampled in high numbers based upon thousandsof logical communications per second and a virtual representation of theposition points PP1-PP8 may be generated based upon the recordedposition points PP1-PP8. Some embodiments may also include a point cloudtype representation a device that comprises the transceivers used torecord position point PP1-PP8, wherein the point cloud representation isbased upon the multiple positions calculated.

Referring now to FIG. 8 , additional apparatus and methods fordetermining a geospatial location and determination of a direction ofinterest may include one or both of an enhanced Smart Device and a SmartDevice in logical communication with wireless position devices 803-810.The importance of geospatial location and determination of a directionof interest is discussed in considerable detail above. As illustrated, aSmart Device 801 may be in logical communication with one or morewireless position devices 803-810 strategically located in relation tothe physical dimensions of the Smart Device. For example, the SmartDevice 801 may include a smart phone or tablet device with a userinterface surface 820 that is generally planar. The user interfacesurface 820 will include a forward edge 818 and a trailing edge 819.

In some preferred embodiments, the Smart Device will be fixedly attachedto a smart receptacle 802. The smart receptacle 802 may include anappearance of a passive case, such as the type typically used to protectthe Smart Device 801 from a damaging impact. However, according to thepresent invention, the smart receptacle 802 will include digital and/oranalog logical components, such as wireless position devices 803-810.The wireless position devices 803-810 include circuitry capable ofreceiving wireless transmissions from multiple wireless ReferencePosition Transceivers 811-814. The wireless transmissions will I tideone or both of analog and digital data suitable for calculating adistance from each respective reference point 811-814.

In some embodiments, the smart receptacle 802 will include a connector815 for creating an electrical path for carrying one or both ofelectrical power and logic signals between the Smart Device 801 and thesmart receptacle 802, For example, the connector 815 may include amini-USB connector or a lightning connector. Additional embodiments mayinclude an inductive coil arrangement for transferring power.

Embodiments may also include wireless transmitters and receivers toprovide logical communication between the wireless position devices803-810 and the Smart Device 801. Logical communication may beaccomplished, for example, via one or more of: UWB, Bluetooth, ANT, andinfrared media.

Reference Position Transceivers 811-814 provide wireless transmissionsof data that may be received by wireless position devices 803-810. Thewireless transmissions are utilized to generate a position of therespective wireless position devices 803-810 in relation to theReference Position Transceivers 811-814 providing the wirelesstransmissions to the wireless position devices 803-840. The wirelessposition devices 803-810 are associated with one or more of: a positionin a virtual model; a geographic position; a geospatial position in adefined area, such as Structure; and a geospatial position within adefined area (such as, for example a Property).

According to the present invention, a Smart Device may be placed into acase, such as a smart receptacle 802 that includes two or more wirelessposition devices 803-810. The wireless position devices 803-810 mayinclude, for example, one or both of: a receiver and a transmitter, inlogical communication with an antenna configured to communicate withReference Position Transceivers 811-814. Communications relevant tolocation determination may include, for example, one or more of: timingsignals; SIM information; received signal strength; GPS data; raw radiomeasurements; Cell-ID; round trip time of a signal; phase; and angle ofreceived/transmitted signal: time of arrival of a signal; a timedifference of arrival; and other data useful in determining a location.

The Nodes 803-810 may be located strategically in the case 802 toprovide intuitive direction to a user holding the case 802, and also toprovide a most accurate determination of direction. Accordingly, aforward Node 803 may be placed at a top of a Smart Device case and arearward Node 804 may be placed at a bottom of a Smart Device case 802.Some embodiments each of four corners of a case may include a Node 805,806, 807, 808. Still other embodiments may include a Node 609 and 810 oneach lateral side.

Referring now to FIG. 9 , in some embodiments, Nodes 903A-910A may beincorporated into a Smart Device 901A and not require a smart receptacleto house the Nodes Nodes 903A-910A that are incorporated into a SmartDevice, such as a smart phone or smart tablet, will include internalpower and logic connections and therefore not require wirelesscommunication between the controller in the Smart Device 901A and theNodes 903A-910A. Nodes 903A-910A may communicate with multiple positiontransponders 911-914.

A Smart Device 901A with integrated Nodes 903A-910A and a Smart Device901A with Nodes 903A-910A in a smart receptacle 902A may provide adirectional indication, such as a directional Vector 917A, withoutneeding to move the Smart Device from a first position to a secondposition.

Referring now to FIG. 10 , method steps are described for determining anAgent position location. At step 1022 a transmission of a layer onelocation designation may be performed of at least one Agent supportedposition transceiver. At step 1023 a receipt of a designation of a layertwo position reference transceiver based upon the layer one location mayoccur. At step 1024, a calculation of a distance from multiple layer twoposition reference transceivers and at least one Agent supportedtransceiver may occur. At step 1025, the operations of step 1024 may berepeated for multiple calculated distances until a position of the Agentsupported transceiver may be calculated based upon these calculateddistances. At step 1026, again measurements may be repeated multipletimes where the same frequency is utilized. At step 1027, again themeasurement steps may be repeated multiple times; however, utilizingdifferent frequencies. At step 1028, the various data may be used tocalculate an estimated position of the at least one Agent supportedposition transceiver based upon a mathematical process. At step 1029, avector may be generated based upon at least one of the positions of theAgent supported transceiver based upon the calculated distances and theestimated position of the at least one Agent supported positiontransceiver.

Referring to FIG. 11 , at step 1132, a location of a first positiontransceiver supported by an Agent is calculated based upon wirelesscommunication with a reference position transceiver. Next, at step 1133,a location of a second position transceiver supported by the Agent iscalculated based upon wireless communication with a reference positiontransceiver. At step 1134, a vector is generated based upon the locationof the first position transceiver and the location of the secondposition transceiver. At step 1135, a database is queried based on thevector. At step 1136, a user interface is generated based on a responseto the query based on the vector. Finally, at step 1137, the userinterface is displayed on a smart device associated with the Agent.

In some embodiments of the invention, determining the location ofobjects in a Cold Storage Facility may benefit from an Augmented VirtualModel. The present invention references prior applications and issuedpatents owned by the applicant relating to automated apparatus andmethods for generating improved Augmented Virtual Models (sometimesreferred to herein as an “AVM”) of a Structure. An AVM may include AsBuilt elements, including the locations of Reference PositionTransceivers and items of interest to which an Agent may wish to beguided by methods and apparatus described herein.

An AVM may include original design data matched to As Built datacaptured via highly accurate geolocation, direction and elevationdetermination. As Built data is matched with a time and date of dataacquisition and presented in two- and three-dimensional visualrepresentations of the Property. The augmented models additionallyinclude data relating to features specified in a Property design anddata collected during building, deployment, maintenance, andmodifications to the Property. In some embodiments, a fourth dimensionof time may also be included. As a result, an Employee or other Agentmay be guided to, for example, a former location of an item of interest.

The experience of the physical Structure is duplicated in the virtualAugmented Virtual Model, The AVM may commence via an electronic modelgenerated via traditional CAD software or other design type software. Inaddition, the AVM may be based upon values for variables, including oneor more of: usage of a Structure; usage of components within theStructure; environmental factors encountered during a build stage ordeployment stage; and metrics related to Performance of the Structure.The metrics may be determined, for example, via measurements performedby sensors located in and proximate to Structures located on theProperty.

Referring now to FIG. 4 , based upon the calculated location of the userdevice 1206. details of the physical Structure 1202A may be incorporatedinto the virtual structure 1202B and presented to a user via a graphicaluser interface (GUI) on the user's Smart Device 1206. For example, auser may approach a physical Structure and activate an app on a SmartDevice 1206. The app may cause the Smart Device 1206 to activate a GPScircuit included in the Smart Device and determine a general location ofthe Smart Device 1206, such as a street address designation. The generallocation will allow a correct AVM 1200 to be accessed via a distributednetwork, such as the Internet. Once accessed, the app may additionallysearch for one or more Reference Position Transceivers 1221A of a typeand in a location recorded in the AVM. An AVM may indicate that one ormore RFID chips are accessible in certain rooms in the Structure. Theuser may activate appropriate sensors to read the REID chips anddetermine their location. In another aspect, an AVM 1200 may indicatethat Reference Position Transceivers 1221A are placed at two or morecorners (or other placement) of a physical Structure 1202A and each ofthe Reference Position Transceivers 1221A may include a transmitter witha defined location and at a defined height. The Smart Device 1206, orother type of controller, may then triangulate with the ReferencePosition Transceivers 1221A to calculate a precise location and heightwithin the physical Structure as discussed above.

Similarly, a direction may be calculated via a prescribed movement ofthe Smart Device 1206 during execution of code that will record a changein position relative to the Reference Position Transceivers 1224A. Forexample, a user Smart Device 1206, such as a smart phone or tablet maybe directed towards a wall or other Structure portion and upon executionof executable code, the Smart Device may be moved in a generallyperpendicular direction towards the wall. The change in direction of theuser device 1206 relative to the Reference Position Transceivers 1221Amay be used to calculate a direction. Based upon a recorded positionwithin the Structure 1202A and the calculated direction, a data recordmay be accessed in the AVM 1200 and a specific portion of the AVM 1200and/or the virtual structure 1202B may be presented on the Smart Device1206. In other embodiments, a direction may be chosen, or verified via amechanism internal to the Smart Device 1206, such as a compass oraccelerometer.

Embodiments may include models generated using, for example, standardmodeling software such as BIM 360™ field which may support the displayof a Structure design in a very complete level of detail. Additionalevents for scanning may occur during the construction process to captureaccurate, three-dimensional as-built point-cloud information. Pointcloud may include an array of points determined from image captureand/or laser scanning or other data collection technique of as-builtfeatures. In some examples, captured data may be converted into a 3Dmodel, and saved within a cloud-based data platform.

In some examples, other methods of capturing spatially accurateinformation may include the use of drones and optical scanningtechniques, which may include high-resolution imagery obtained frommultiple viewpoints. Scanning may be performed with light-based methodssuch as a CCD camera or LIDAR. Other methods may include infrared,ultraviolet, acoustic, and magnetic and electric-field mappingtechniques may be utilized.

Structure-related information may include physical features generallyassociated with an exterior of a Structure such as geolocation,elevation, surrounding trees and large landscaping features, undergroundutility locations (such as power, water, sewer, sprinkler system, andmany other possible underground utility features), paving, and pool orpatio areas. Structure-related information mini also include featuresgenerally related to a Structure such as underground plumbing locations,stud locations, electrical conduit and wiring, vertical plumbing piping,and HVAC systems or other duct work. The acquisition of the data mayallow the model system to accurately locate these interior and exteriorfeatures. Acquisition of as-built data during different points of theconstruction completion allows measurements to be taken prior to aspectsinvolved in a measurement process being concealed by concrete, drywallor other various building materials.

Referring to FIGS. 13A-13D, an illustration of the collection of data byscanning a Structure during its construction is provided. In FIG. 13A, adepiction of a site for building a Structure is illustrated. Thedepiction may represent an image that may be seen from above the site.Indications of Property boundaries such as corners 1301 and Propertyborders 1302 are represented and may be determined based onsite scanningwith Property markings from site surveys or may be entered based onglobal coordinates for the Property lines. An excavated location 1303may be marked out. Roadways, parking and/or loading areas 1304 may belocated. Buried utilities such as buried telephone 1305, buried electric1306, buried water and sewer 1307 are located in the model asillustrated. In some examples, other site services such as a buriedsprinkler system 1308 may also be located.

Referring to FIG. 13B, the excavated location 1303 may be scanned orimaged to determine the location of foundation elements. In somenon-limiting examples, a foundational footing 1321 along with buriedutilities 1322 is illustrated. The buried utilities may includeutilities such as electric lines, water supply (whether from a utilityor a well on-location), sewer or septic system lines, andtelecommunications lines such as telephone, cable and internet. Otherfooting elements 1323 may be located at structural requiring locationsas they are built. In some examples, a scanning system may provide thelocational orientation relative to site-orientation markings. In otherexamples, aerial imagery such as may be obtained with a drone may beused to convert features to accurate location imagery.

Referring to FIG. 13C, a wall 1331 of the Structure in the process ofbuild is illustrated. The Structure may be scanned by a scanning element1330. In some examples, a laser three-dimensional scanner may be used.The wall may have supporting features like top plates 1333, headers1336, studs 1332, as well as internal items such as pipes 1334,electrical conduits, and wires 1335. There may be numerous other typesof features within walls that may be scanned as they occur such as airducts, data cables, video cables, telephone cables, and the like.

Referring to FIG. 13D, the wall may be completed with Structurecomponents behind wall facing 1340 may no longer be visible. Electricaloutlets 1341 and door Structures 1342 may be scanned by a scanningelement 1330.

Referring to FIG. 13E, a wireless Node may be fixedly attached to aposition in or proximate to a Structure. In some embodiments, attachmentmay be accomplished during, construction and/or retrofitting of astructure, in which case the functionality described herein may be madeoperational to track Agents, materials, equipment, and the like during aconstruction phase, and also track a location of materials and equipmentincluded, in the structure. Nodes may be installed as Reference PositionTransceivers or be attached to items that dynamically change positions,such as, by way of non-limiting, example one or more of: Agents,building materials, structural components, electrical components,plumbing components, equipment, machines and architectural aspects (e.g.a corner, an arch, an extremity, and the like).

In some non-limiting examples of a wireless Node, a Bluetoothcommunications hub compatible with a standard such as, for exampleBLE5.1 (Bluetooth Low Energy 5.1) or Wi-Fi RTT may be fixedly attachedto a structural component, such as a door header 1336 as Node 1350acting as a Reference Position Transceiver. In another example, a Node1351 may act as a Reference Position Transceiver and be attached to awall stud, preferentially one that has electrical conduit 1335 runningalong it. In some embodiments, the electrical conduit 1335 may supplypower to the Node 1351. Alternatively, a Node 1352 may be configured aspart of a receptacle box. In some examples, one or more Nodes 1350-1351may be battery powered. One or more Nodes 1350-1351 may be powered viaelectrical supply wiring 1353 from a nearby power conduit 1335 so thatthe Node 1350-1351 may be tied into a centrally powered electricalsystem. Moreover, the Nodes may be adapted to de-power and de-couplefrom a network based on a power supply status or a power drain change.

Referring to FIG. 13F, an illustration of an Agent 1370 utilizing anoriented stereoscopic sensor system 1371 to orient a direction ofinterest is shown. The stereoscopic sensor system 1371 may obtain twodifferent images from different viewpoints 1372-1373 which may be usedto create topographical shape profiles algorithmically. A controller mayobtain the image and topographic data and algorithmically compare themto previously stored images and topographic data in the generalenvironment of the user. The resulting comparison of the imagery andtopography may determine an orientation in space of the Agent 1370 andin some examples determine a device field of view. The controller mayutilize this determined field of view for various functionality asdescribed herein.

With the methods and apparatus described above, the location of anindividual with a Smart Device or other Node may be determined.Additionally, the methods and apparatus may be used to determine theindividual's direction of interest. Similarly, the location of anyobject tagged with a transceiver, Node, or Smart Device may bedetermined, whether stationary or in transit. In a Cold StorageFacility, equipment, sensors, pallets, transports, and employees mayhold or be “tagged” with a transceiver, Node, or Smart Device, and eachlocation may be transmitted to monitoring system and/or identified on auser display to monitor the location of each item. Additionally, theitem or persons may transmit to a system information about theitem/person, either automatically or when queried. For example, thesensors may transmit location and also data for the sensor over anydesired time period. Equipment and sensors may be able to transmitinformation about maintenance history or part numbers. Movement ofpallets and transports may be tracked throughout the Facility. Employeemovements may also be tracked and information about the Employee,including hours, authorizations, and the like, may be transmitted to amonitoring system. These are non-limiting examples and other types ofinformation may be transmitted from the Node or Smart Device to amonitoring system.

In some cases, it may be prohibitively inefficient or expensive to “tag”or associate a Node or Smart Device with every item to be monitored. Forexample, it may be cost and time prohibitive to assign each productbeing stored in the Facility with a Node or Smart Device. As such, othermethods of monitoring the products and other items may be used. In someembodiments of the invention, an item may be assigned a “Virtual Tag,”in which a Smart Device can identify an object in its field of view andcapture its location coordinates based on the position of the SmartDevice. A “Virtual Tag” or “icon” may be generated in the Smart Device'sfield of view at those coordinates and saved in the digital memory.Information regarding the object may also be saved with the spatialcoordinates. This information can be transmitted to the monitoringsystem so that others may find the object by routing to the particularcoordinates. In addition, when that Smart Device, or another SmartDevice in logical communication with the monitoring system, views theparticular location, the Smart Device will identify that a virtual tagexists at that location with, for example, an icon. The icon may also beinteractive so that information associated with the object may be known.Thus, the monitoring system may be in logical communication with bothphysical tags such as Smart Devices or Nodes and with Virtual Tagscreated by a Smart Device. This will be described in further detailbelow.

Creating of Virtual Tags may, in sonic embodiments, performed using thefollowing steps, Referring to FIG. 14 , in a first step 1401, anorientation home position is established. At step 1402, the Smart Deviceinitiates transceiving and at step 1403, a coordinate system isselected. If Cartesian coordinate system is selected, at step 1404, theSmart Device may triangulate with Reference Point Transceivers/Nodes todetermine the position of the Image Capture Device in the Smart Device(step 1406). If a polar coordinate system is selected, at Step 1505, anangle and distance involved in transceiving is measured, and the cameraposition is determined (step 1406). In some embodiments, the ImageCapture. Device may be a camera, including a charge-coupled device(CCD), or a LIDAR apparatus, including a solid-state LIDAR or aMEMS-based LIDAR

At step 1407, the position of any Nodes or Reference Point Transceiversis verified, and in step 1408, the direction of the Image Capture Deviceis determined based on the methods described herein, using the SmartDevice and any Nodes or Reference Point Transceivers nearby.

At step 1409, the Image Capture Device captures an image. This may bedone by capture of wireless energy of a first wavelength is receivedinto a wireless receiver in the Smart Device. In exemplary embodiments,this step may comprise receiving image data based on visible light intoa camera of the smart device. The wireless energy may be dispersed overa one-, two-, or three-dimensional space in a defined physical area, andmay be received into a one-, two-, or three-dimensional array in thereceiver. The wireless energy may take the form of electromagneticradiation, such as light in the human-visible light spectrum (generallyhaving a wavelength between 380 nm-740 nm), ultraviolet light (generallyhaving a wavelength between 10.0 nm-400 nm), or infrared light(generally having a wavelength between 740 nm-2.00 mm). The set ofwireless energy available to the wireless receiver is the smart device'sField of View. In step 1410, the Field of View may change as the angleor position of the Smart Device changes.

In step 1411, the Field of View is quantified. In some embodiments, apattern of digital values is generated based upon receipt of wirelessenergy into the wireless receiver. This pattern of digital values may bebased on one or more qualities of the received wireless energy,including its intensity, spatial dispersion, wavelength, or angle ofarrival. The pattern may be placed into an appropriate array. Forexample, if the display of the smart device is a two-dimensionaldisplay, then the pattern of digital values may comprise atwo-dimensional representation of the image data received. In sonicembodiments, the pattern of digital values may be based on an aggregatedset of values from an array of receivers. For example, if the basis ofthe digital values is the intensity of the wireless energy received intothe receiver, then the digital value assigned to a given entry in thearray may be based on a weighted average of intensity of wireless energyreceived at a plurality of the receivers in the array. The set ofdigital values within the Field of View is the Digital Field of View. Instep 1412, optionally, the coordinates of the space in the Digital Fieldof View may also be determined. Any suitable method may be used, but insome embodiments, LIDAR may be used to determine the presence anddistance from the Smart Device of various items in the Field of View andtherefore determine the position of items. In step 1413, items or spacein the Field of View may be “tagged” such as via a touch screen orcursor. Using similar steps for step 1412, the location coordinates forthe Virtual Tags may be known and transmitted, for example, to amonitoring system or controller.

Referring to step 1414 when a Smart Device that is part of themonitoring system approaches the location of the Virtual Tag, the SmartDevice will identify that the Virtual Tag is within the Smart Device'sField of View. In step 1415, When this occurs, the Smart Device willgenerate an icon in the position of the Virtual Tag. This icon may beinteractive so that information regarding the location or Virtual Tagmay be ascertained. For example, the Virtual Tag may provide a locationand environmental history of the product at the coordinates, as will bediscussed in further detail below.

Next, additional details regarding how the Field of View may bedetermined will be discussed. Referring now to FIG. 15 , a Smart Device1501 is illustrated in within a wireless communication area (WCA) 1502.The WCA 1502 may be dimensioned according to a bandwidth and/or modalityof wireless communication the Smart Device 1501 uses to transmit andreceive information. For example, bandwidths may include thoseassociated with UWB, WiFi, Bluetooth, ANT, ultrasonic, infrared andcellular modalities of communication. In general, unless otherwisecontained by physical modification such as a directional antenna orradio frequency interference from a physical object (such as objectswith significant metallic content; objects with high water content;electrical fields, etc.), a WCA 1512 will include spherical area(s)emanating from one or more transceivers and/or transceiver antennasoperated by the Smart Device 1501.

As discussed extensively herein, the location of the Smart Device 1501is determined based upon wireless coordinates to and/or from the SmartDevice 1501; and described via a coordinate system, such as Cartesiancoordinates, or other X,Y,Z coordinates, polar coordinates, sphericalcoordinates, and cylindrical coordinates. Aspects of wirelesscommunications that may be referenced to generate location coordinatesmay include one or more of: RTT (round trip time), time of flight, RSSI(received signal strength indicator); angle of an angle of departure,and other aspects described herein.

With the location of the Smart Device 1501 determined, a location of theWCA 1502 may be extrapolated based upon the location of the Smart Deviceand a transceiving distance the Smart Device is capable of. According tothe present invention, a portion of the WCA 1512 is selected as a radiotarget area (RTA) 1512 from which the Smart Device 1501 may receivespecific bandwidths of electromagnetic radiation. In preferredembodiments, the RTA 1512 will include a frustum expanding outward in aconical shape from one or more energy receivers 1509 included in theSmart Device 1501. The frustum Shaped RTA 1512 will overlap a portion ofthe WCA 1502 sphere. Other shapes for a radio target area 1502 are alsowithin the scope of the invention. In some embodiments, a shape of theRTA 1512 will be based upon receiving capabilities of the one or moreenergy receivers 1509. For example, an energy receiver 1502 with acharge coupled device (CCD) or complementary metal oxide semiconductor(CMOS) receiver may have a single plane receiving surface and be bestmatched with a frustum of a generally conical shape, an energy receiver1509 with multiple receiving surfaces (e.g. with multiple CCD and/orCMOS devices) may be arranged to enable a more complex shaped RTA 1512.

In some preferred embodiments, a direction of interest 1511 winintersect the RTA 1512. As discussed herein, the direction of interest1512 may be represented by a ray or vector 1511. In addition, thedirection of interest may be represented as a direction of interestarea, such as a frustum 1511C defined by multiple rays or vectors 1511,1511A, 1511B. In various embodiments, the direction of interest 1511area may encompass the RTA 1512 or be a subset of the RTA 1512.

A direction of interest may be determined for example via the methodsand devices described herein and may be associated with a directionbased upon a ray or vector indicative of a direction of interest 1511, adirection based upon a magnetic field sensor, an accelerometer, a lightbeam, correlation between two tags or nodes, Agent gestures, or otherSmart Device recognized apparatus and/or method.

One or more transceivers 1503-1504 (typically included within a SmartDevice, Tag, or Node) will be located within an area defined by the RTA1512. According to the present invention, a position of the transceiver1503-04 will be determined and a user interactive mechanism will begenerated at a position of the transceiver 1503-04 within a graphicaluser interface emulating aspects of the RTA 1512 on the Smart Device1501 or another user interactive interface screen (not shown, and maybeat a site remote to the RTA 1512).

According to the present invention, some portion of the RTA 1512 (whichmay include the entirety of the RTA 1512) may be portrayed on an Agentinterface 1510, including, in some embodiments, a human readablegraphical user interface (GUI), The interface 1510 may include arepresentation of a particular level of electromagnetic energy receivedvia the energy receiver 1509 and associated with a particular area ofthe RTA 1512. For example, energy levels of an infrared wavelength thathas emanated from or reflected off of an item in the WTA 1512 andreceived via an infrared receiver in the Smart Device 1512 may be usedto generate a heat map type interface display, similarly energy that hasemanated from or reflected off of an item in the RTA 1512 in the 400 nMto 700 nM range and been received via a charged couple device in theSmart Device 1501 may be portrayed as a human visible image of items inthe area included in the RTA 1512. Other embodiments may include a pointcloud derived from electromagnetic energy bouncing off of or emanatingfrom items included in the RTA 1512 or a series of polygons generatedbased upon a LIDAR receiver in the Smart Device 1512. An Agent interface1510 may be presented in a modality understandable to an Agent type. Forexample, an interface presented to a UAV air UGV may include a digitalpattern and an interface presented to a human Agent may include multiplepixels or voxels generating a pattern visible to a human being.

The wireless location methods and apparatus described herein may bedeployed in conjunctions with one or more Tags and/or Nodes 1503-08located with the WCA 1502 to generate location coordinates for the Tagsand/or Nodes 1503-08. A controller or other device operating a processormay determine which Tags and/or Nodes 1503-08 are located within thethree dimensional space included in the RTA 1512 based upon a) thelocation of the Tags and/or Nodes 1503-08; and b) the location of areaincluded in the RTA 1512.

In another aspect of the present invention, in some embodiments, someenergy levels may not be represented in the Agent interface 1510. Forexample, in some embodiments, energy levels reflected off of aparticular item may not be included in the Agent interface 1510 whichwould a simulation that the item is not within the RTA 1512. Otherembodiments may only represent energy levels that have reflected off ofselected items within the RTA 1512 thereby emphasizing the presence ofthe selected items and ignoring the presence of other items within theRTA 1512.

In some embodiments of the invention, sensors throughout the facilitymay be virtually tagged via a Virtual Tag method described herein orfrom an AVM using As Built data. As such, throughout the facility, whena Smart Device views the sensor, an icon may be generated. When a userinteracts with the icon, the icon may generate information about thesensor or equipment. For example, the following equipment could bevirtually tagged: refrigeration compressors; refrigeration pumps;refrigeration valve groups; refrigeration evaporative condensers and fancoil units; air compressors; boilers; and electrical panels. Examples ofsensors that could be virtually tagged include sensors that detecttemperature, humidity, air flow, current/amperage, pressure, water flow,water sensor, and ammonia levels. When icons associated with suchequipment or sensors are generated, a user may interact with the icon,and information about the items may be displayed on the smart device.For example, when touched by a user, the icon may bring up a menu of theinformation about the equipment or sensor, including a user's manual,parts list, maintenance history, and/or history of data generated by thesensor or equipment over a certain time period. The icons may alsoprovide an indication or warning if there is a problem with theequipment or sensor.

As a non-limiting example, ammonia sensors may be installed in variouslocations in the cold storage facility, including the refrigerationmachine room, along the ammonia piping, near valve groups, at the fancoils, within ductwork (if applicable), and the like. A smart device mayprovide an AVM of the facility that identifies each ammonia sensor, andif the icon for each sensor is opened, information about the particularsensor may be obtained, including the ammonia detection over time (for apredetermined time period), maintenance history for the sensor, partnumbers, and user manual. In addition, if an ammonia sensor shows areading outside a predefined ranged deemed acceptable, a warning oralert may be generated and the Agent/Employee may be notified. Inaddition, the icon itself may generate a label identifying the sensorwith the readings outside the predefined concentration range.Additionally, the icon may allow for various alerts to be generated onthe icon so that the urgency of the problem may be understood by theEmployee. For example, different colors may denote different levels ofurgency, such as yellow denoting a mild increase about the predefinedconcentration range, orange denoting a higher concentration, and redindicating a dangerously high level of ammonia detected. Of course, anycolor combination may be used, and other methods of identifying theurgency with the problem may be used including increase in intensity ofan icon, increase in sound or frequency of an alert, or a flashing icon(with an increase in flashing frequency indicating increasing urgency).Additionally, the orienteering methods may allow for an Agent to locatethe problematic sensor, either via the smart device or a heads updisplay associated therewith.

Similarly, this technology may be useful with detectors for carbonmonoxide, fire protection leaks, fire protection water tank storagelevels and/or water quality, and hazardous materials in areas such aschemical storage rooms. Many other types of sensors and processes can bemonitored with using smart devices by such methods. In addition tosensors and equipment in the facility, virtual tags may allow for anEmployee to locate key underground features in U/G piping, stormwater,process waste, sanitary, etc., such as turns in pipes, lift stations,manholes, etc. This would allow for isolation of the 3D model of theunderground systems and display using augmented reality using a smartdevice or head's up display and using the location sensors to “anchor”the model, so that it's always accurate.

Another example is the use of virtual or physical tags for moisturedetection, such as moisture sensors in duct and pipe insulation(underside, low points, etc.), including those installed indoors oroutside, that are subject to condensation and leaks. Based on active‘warnings’ from the smart device, an Agent can locate the condensationissues using the orienteering methods described herein using a smartdevice or head's up display.

Referring now to Figs. 16A-16G, exemplary Wireless Communication Areas(WCA) and Radio Target Areas (RTAS) are illustrated. In general, a WCAis an area through which wireless communication may be completed. A sizeof a WCA may be dependent upon a specified modality of wirelesscommunication and an environment through which the wirelesscommunication takes place. In this disclosure (and as illustrated), aWCA may be portrayed in a representative spherical shape; however, in aphysical environment, a shape of a WCA may be amorphous or of changingshape and more resemble a cloud of thinning density around the edges, Ingeneral, a RTA is an area from which an energy-receiving Sensor willreceive energy of a type and bandwidth that may be quantified by theenergy-receiving Sensor. The RTA shape and size may be affected by anenvironment through which the energy must be conveyed and furthereffected by obstructions.

According to the present invention, a WCA and RTA may be used to providean augmented reality user interface indicating a location of variouspersons within the WCA and RTA and indications of sensor readingsrelated to those persons as well as links to further information.

Referring now to FIG. 16A, a side view illustrates a WCA 1600surrounding a Node, such as a Smart Device 1602. Energy 1603, which isillustrated as rays, is received by one or more energy-receiving Sensors1604 in the Smart Device 1602 (energy-receiving Sensors may also be in aSmart Receptacle associated with the Smart Device, though this is notillustrated in FIG. 16A). An exemplary ray 1603 proceeds from a position1605 within RTA 1601 boundary to the energy-receiving Sensor 1604.

As illustrated, a portion 1601 a of the RTA 1601 may flatten out inresponse to a ground plane, wall, partition, or other obstructionencountered. A Node 1606 may be located on or within a surface thatmakes up a relevant obstruction and the Node 1606 may appear to be alonga perimeter of the RTA 1601. Similarly, a Virtual Tag may be associatedwith location coordinates that appear on or within a floor, wall,partition, or other article acting as a radio frequency obstruction andthereby appear to be a part of the obstruction, however, since it isvirtual, the Virtual Tag will not effect the physical properties of theobstruction. Essentially, a Virtual Tag may have location coordinatesthat correspond to anywhere in the physical real-world. In someexamples, a software limit or setting may limit location coordinates ofVirtual Tags to some distance from a base position or a distance from adesignated position, such as a location of a designated Physical Tag,Reference Point Transceiver or other definable position.

In addition to obstructions, a topography of an environment within anRTA 1601 may also limit wireless conveyance of energy within an RTA 1601to an energy-receiving Sensor 1604. Topography artifacts may include,for example, a terrain, buildings, infrastructure, machinery, shelvingor other items and/or other structures that may create impediments tothe receipt of wireless energy.

Energy received 1603 into the energy-receiving Sensor 1604 may be usedto create aspects of a user interface that is descriptive of theenvironment within the RTA 1601. According to the present invention,environmental aspects, Nodes 1606, Tags (both physical Tags and VirtualTags) and the like may be combined with user interactive mechanisms,such as switches or other control devices built into a user interactivedevice, and included in a user interactive interface. For example,energy levels received into an energy-receiving Sensor 1604 may becombined with location coordinates of Physical Tags and/or Virtual Tagsand a user interactive device may be positioned in an interactive userinterface at a position correlating with the position coordinates and besurrounded with a visual indicator or the received energy levels.

In this manner, a single user interface will include a static imagerepresentative of received energy levels at an instance in time; avisual representation of a location(s) of Physical and/or VirtualTag(s), and devices with user interactive functionality. In sonicembodiments, the devices with user interactive functionality may bepositioned at a location in the user interactive interface correlatingwith the position(s) of the Physical and/or Virtual Tag(s).

This disclosure will discuss RTAs 1601 that are frustums of a generallyconical shape, however, RTAs 1601 of other volume shapes are within thescope of the invention. For example, if an energy-receiving Sensor 1604included a receiving surface that was a shape other than round, or hadmultiple receiving surfaces, each of a round or other Shape, the RTA1601 associated with such an energy-receiving Sensor 1601 may have ashape other than a frustum of generally conical shape.

Referring now to FIG. 16B, a top-down view of a RTA 1601 is depicted. ARTA 1601 will include sonic portion of a WCA 1600. As illustrated, theWCA 1600 includes a space with irregular boundaries encompassing 360degrees around the Smart Device 1602. Aspects such as topography,strength of signals and atmospheric conditions (or other medium throughwhich a wireless communication will travel) may affect and/or limit aperimeter of the WCA 1600. A location of the RTA 1601 may be referencedto determine which Tags (Physical and/or Virtual) such as node 1606 areincluded within an interactive user interface. Generally, preferredembodiments may only include Tags with location coordinates with the RTA1601 in the interactive user interface. However, embodiments may includeTags external to the RTA 1601 in a particular interactive userinterface.

Referring now to FIG. 16C, a side view of a WCA 1600 is presented wherean energy-receiving Sensor 1604 is capable of quantifying a particularform of energy, such as a particular bandwidth of energy received from auser selected RTA 1607. A Smart Device 1602 may incorporate or be inlogical communication with multiple energy receiving devices 1604, eachenergy receiving device capable of quantifying a limited energy spectrumin an environment defined by the RTA 1607 selected by the user.

Some embodiments include a RTA 1607 that varies according to a type ofenergy receiving device 1604 receiving a corresponding type of energy.For example, an energy-receiving Sensor 1604 that receives energy in alower bandwidth may have an RTA 1607 that extends a greater distancethan an energy-receiving Sensor 1604 that receives energy in a higherbandwidth. Similarly, some energy-receiving Sensors 1604 may be effectedby farces outside of the RTA 1607, such as a magnetometer which may besensitive to signal interactions around all of the WCA 1600, and a RTA1607 associated with a magnetometer may accordingly be the same as theWCA 1600.

By way of non-limiting example, a RTA 1607 for a CCD-type energyreceiver may be represented as a frustum with an expansion angle ofapproximately 60 degrees in shape. Accordingly, the RTA 1607 subtendsonly a portion of the universal WCA 1620.

Referring now to FIG. 16D, a top view of a WCA 1600D is illustrated witha RTA 1607A comprising a frustum with an expansion angle ofapproximately 60 degrees. A Smart Device with an energy receiver 1602that quantifies a specified bandwidth of energy from the RTA 1607A maygenerate a user interface with an image based upon energy quantifiedfrom RTA 1607A.

In FIG. 16D, the WCA 1600D is represented as a spherical area. A WCA1600D may be designated that is less than an entire area of possibleradio communication using a specific designated wireless communicationmodality. For example, WCA 1600D may be spherical and stay withinboundaries of a modality based upon a UWB wireless communicationprotocol.

A user interface based upon quantified energy in an RTA 1607, 1607A, maypresent a representation of energy within the respective RTA 1607, 1607Aas quantified by an energy-receiving Sensor 1602. Energy levels of otherthree-dimensional spaces within the WCA 1600 may be quantified by energyreceivers and presented in a user interface by directing energy from aselected three-dimensional space into the energy receivers and therebydefining a different RTA. In this manner, energy levels may bequantified from essentially any area within the WCA 1620 and representedas part of a user interface. Quantified energy levels may vary basedupon a receiving Sensor. For example, a CCD Sensor may quantify visiblelight spectrum energy, and a LIDAR receiver a broad spectrum, aninfrared receiver may quantify infrared energy levels, andenergy-receiving Sensors. A particular feature present in a particularportion of the electromagnetic spectrum quantified by anenergy-receiving Sensor may have a unique physical shape whichcharacterizes it and which may be associated with a correspondingvirtual-world aspect and Tag associated with the location.

In some examples, as has been described, quantification of energy levelsassociated with aspects of the physical world may be for one or more of:characterizing an RTA 1607, 1607A by quantifying energy levels andpatterns existing at an instance in time, determining a location and/ororientation of a Smart Device 1602 or other Node, such as node 1606; andverifying a location and/or orientation of a Smart Device. In someexamples, energy levels associated with aspects of the physical worldmay be communicated by the Smart Device to a remote controller forfurther processing, and the remote controller may communicateinformation back to the Smart Device or to another user interface.Information communicated from the controller may include, for example,an orientation of physical and/or virtual aspects located within theuniversal RTA in relation to the Smart Device; quantified energyindicating of or more of: a topographical feature, a surfacetemperature, a vibration level, information associated with a VirtualTag, information associated with a physical Tag, sensor data, or otherinformation associated with the RTA 1607A.

A view of a Radio Target Area 1607A may be a relatively small portion ofthe entire WCA that surrounds a Smart Device. An area of energy to bequantified by a sensor (sometimes referred to herein as the Radio TargetArea) may be displayed surrounded by the WCA 1630.

Referring now to FIG. 16E, an exemplary presentation of a RTA 1644superimposed upon a representation of a WCA 1641 is illustrated. The WCA1641 is illustrated with a perspective view of a spheroid with analignment feature 1660 such as a spheroid dividing arc, or a line. Ablackened ellipsoid feature is a representation of the RTA 1644associated with a particular Smart Device which would be located at acenter of the spheroid WCA 1641. If desired, one or more energyreceiving devices associated with or incorporated into a Smart Devicemay be repositioned or have a changed orientation in space to ultimatelyscan all of the accessible universal Radio Target Area space.

Referring to FIG. 16F, an illustration of how moving the one or moreenergy receiving devices around in space may alter an area defined asthe RTA 1654. The same orientation of the universal WCA 1641 may benoted by a same location of the alignment feature 1660. Relativemovement of the ellipsoid feature illustrates a change in an areadesignated as RTA 1654.

Referring to FIG. 16G, an illustration of adding Tag locations which maybe Physical Tags or Virtual Tags) to a mapping of the WCA 1641 isprovided. A Tag may be represented in the WCA, for example, as an icon(two- or three-dimensional) positioned in space according to acoordinate system, such as Cartesian coordinates, polar coordinates,spherical coordinates or other mechanism for designating a position.Coordinates may specify one or both of physical real-world Tags andVirtual Tags.

A location of a real-world Tag or Virtual Tag may be in either RTA 1661,the WCA 1641 or external to both the RTA 1661 and the WCA 1641. Examplesof Tags outside the RTA 1661 and within the WCA 1641 include Tags1662-1666. An example of a Tag in the device RTA is Tag 1661. A Taglocated external to of the WCA 1641 and the RTA 1661 includes Tag 1667.

In some examples, a display on the user's Smart Device may illustrateimage data captured via a CCD included in a Smart Device. Portions ofthe image data captured via a CCD may be removed and replaced with anicon at position correlating to a position in space within the RTA 1661.The icon may indicate of a Tag 1661 located within the RTA 1661, or atleast the direction in the RTA 1664 along which the Tag 1661 may belocated at an instance in time. In addition, an area of a user interfaceportraying the Icon may user interactive device such that when thedevice is activated, the Smart Device is operative to perform an action.

The actual positions of the Tags in real-world space (or the digitalequivalent in the real-world space) may be stored and maintained in adatabase. Positions of physical Tags may be determined via techniquesbased upon wireless communication and be updated periodically. A periodof update may be contingent upon variables including, user preference,Tag movement, change in environmental conditions, User query or othervariable that may be converted into a programmable command. In anotherexample of some embodiment, an Agent may interact with a user interfaceand understand the presence of Tags that are outside of the RTA 1661 andadjust one or both of a position and direction that the Smart Device tocause the Smart Device to be positioned such that the RTA 1661encompasses a position of the Tag of interest.

Referring to illustration FIG. 17A, an exemplary apparatus foreffectuating the methods described herein is shown, wherein Smart Device1701 has within its Radio Target Area a Structure 1721. Smart Device1701 may display a user interface 1702 based upon data generated by anenergy-receiving Sensor 1703 incorporated into the Smart Device oroperative in conjunction with the Smart Device 1701. Theenergy-receiving Sensor 1703 may produce data representative of an areafrom which the energy-receiving Sensor 1703 received energy. A userinterface 1702 may be generated that is based upon relative values ofsome or all of the data produced by the energy-receiving Sensor 1703.

Smart Device 1701 may have its position and direction determined usingthe orienteering methods described herein, with reference to ReferencePoint Transceiver 1731. The position may be determined relative to aBase Node, which may operate as an origin in a coordinate systemassociated with Structure 1721 and its surroundings. Theposition-determination step may be aided with reference to transmitter1722, which in some embodiments, may be a Reference Point Transceiver.In this example, transmitter 1722 is positioned proximate to theStructure 1721.

A receiver on Smart Device 1701 may be operative to receive a wirelesslogical communication from transmitter 1722. This communication may bein one of a variety of modalities, such as Bluetooth, ultra-wideband,radiofrequency, etc. Based upon the signal, Smart Device 1701 maytransmit to a server, a database query based upon a determined set ofcoordinates of transmitter 1722.

the database contains an entry comprising, as a data structure, a set ofcoordinates proximate to the set of coordinates of transmitter 1722,then Smart Device display 1702 may display icon 1712 proximate to thelocation of transmitter 1722, as displayed on Smart Device display 1702,or otherwise on the virtual representation of the shop 1711. In thisway, a user of Smart Device 1701 may be alerted to the presence ofinformation associated with structure 1721 in which the user may beinterested.

In sonic embodiments, icon 1712 may be displayed on Smart Device display1702 only if Smart Device 1701 can transmit appropriate permissions tothe database. For example, icon 1712 may only display if Smart Device1701 is on a certain Wi-Fi network or if the user of Smart Device 1701has input the appropriate credentials, In other embodiments, icon 1712may display on any Smart Device display 1702 (if the Radio Target AreaCone 1713 includes transmitter 1722), but further functionality may bebased upon inputting a password (or, in some embodiments, correctlyinput the answer to a question).

In some embodiments, the appearance of icon 1712 may change based uponan identity of the user or based upon some other dynamic. For example,if the user has a certain UUID, and the database includes a messagespecifically intended for a user with that UUID, then the icon may flashto indicate the presence of a message. This message may be displayedtextually, visually, audibly, or by a hologram. Similarly, the databasemay record each instance in which it is accessed by a query from a SmartDevice. Such a record may include a time stamp. If data related tostructure 1721 has changed since the last time stamp, then icon 1712 mayturn red (for example) to indicate such a change. In addition, digitalcontent may be appended to any content already in the database, such asadditional alphanumeric annotation, an audio file, an image file, avideo file or a story file.

Activation of an interactive user device encompassing icon 1712,additional functionality may be provided to the user of the Smart Device1701. For example, selecting icon 1712 may display information aboutStructure 1721, such as shop hours or discounts. Alternatively,activating the interactive user device associated with icon 1712 maygenerate a control panel, which may allow the user to control aspectsrelating to sensors or other electronics within structure 1721. Forexample, upon confirmation that Smart Device 1701 has the appropriatepermissions, selecting icon 1712 may allow the user to turn off thelights within structure 1721.

The Smart Device 1701 may also display other functional buttons on itsdisplay 1702. In some examples, one such function may be to showdisplays of the sensor RTA 1713 in the context of the universal RadioTarget Area surrounding the user, By activating the functional button,the user may be presented with a set of options to display the universalRadio Target Area.

Referring to illustration FIG. 17B, an example of a means ofillustrating, a RTA 1770 is provided. The display screen of the SmartDevice 1701 may display a number of information components. A similarillustration as FIG. 16G may be included as inset 1775. However, adifferent illustration of the RTA 1770 may be formed by flattening thesurface of the illustrated sphere into a flat depiction where each ofthe surface regions may be flattened into a segment 1771 The RTA 1770may lie illustrated on the flat segments. A Tag or icon 1712 may belocated within the device RTA 1770 showing structure 1711. The icon 1712may also be included in the real time display of a representation ofdata generated by an energy-receiving Sensor. Tags may also be locatedoutside of the RTA 1770. An Agent may move around the Smart Device tolocate an RTA that encompasses Tag 1774.

Referring to FIG. 17C, item 1780, an exemplary display screen 1790 forSmart Device 1701 that may be displayed when a user activates a Tag at alocation outside the RTA 1782 is illustrated. Wien the user activatesthe exemplary Tag 1781, a menu 1785 may display. Amongst the variousinformation such as text, imagery, video content and the like that maybe displayed an identification of the Tag 1786, associated textualinformation and data 1787 as well as functional buttons 1789 may bedisplayed on the user interface and may be used by the user to activateadditional function including new display layers, content integrationand control function such as in a non-limiting sense a control to revertto a previous menu display.

Environmental Control Systems and Methods

The foregoing has been illustrative of the location, position, anddirection of interest being determined by the methods and apparatusabove. Additionally, the objects or persons may be identified andlocated by a physical tag or a Virtual Tag. However, it may also bedesirable to know the environmental conditions throughout the facility,and as such be able to correlate object location with the environmentalconditions at that location.

A first aspect of the Environmental Control System is the creation of athree dimensional (3D) environmental profile, whereby an environmentalparameter of interest is detected throughout a 3D space in the Facility.Any suitable environmental parameter can be monitored in this fashion,but as an example, the creation and monitoring of a 3D temperatureprofile will be discussed. Creating a 3D temperature profile in the coldstorage space may be accomplished by any suitable means.

One approach for creating a 3D temperature profile is to installtemperature sensors throughout the space, thus creating a “temperaturemesh” which can be extrapolated to create a 3D temperature profile.Referring to FIG. 18 , a Cold Storage Facility 1800 that includesproducts 1801 may include temperature sensors 1805 on the scaffolding1810, on pallets 1820, on other surfaces 1830 such as walls and floors,and/or suspended 1840 in the air. Additional sensors may be worn byemployees, for example, on the front of a hard hat. The data regardingtemperature over time for each sensor may be transmitted to the system'scontroller, which may allow for the system to create a 3D profile of thearea by extrapolating the data between sensors. Known methods forextrapolating data including those which take into account air flow,humidity, and the like, may be used to create the 3D temperatureprofile. The same approach could be used, for example, with moisturesensors so that data from a plurality of moisture sensors may beextrapolated to create a 3D moisture/humidity profile. Otherenvironmental sensors (e.g., light, air flow, vibration, etc.) may bealso be used to create 3D profiles in a similar manner. In someembodiments, a mesh of barometric or air flow sensors may be situatedthroughout a portion or all of the Facility, whereby the barometricpressure or air flow measurements may be extrapolated to create a 3Dpressure profile in the Facility, which may provide additionalinformation regarding air flow in the Facility.

Another approach to creating a 3D temperature profile is to overlay apositioning or visualization rendering (e.g., from LIDAR) with atemperature profile obtained with a scanning technology such as thermalinfrared imaging. Any suitable way of creating the positioning orvisualization rendering, including video methods, may be used. However,as an example, LIDAR will be discussed. In some embodiments, referringto FIG. 19 , in some embodiments, a LIDAR detector and thermal imagingdevice (not shown) may have a same or overlapping Field of View 1800. Assuch, both the LIDAR detector and the thermal imaging device may capturedifferent information about the same geospatial location. For example,as shown in FIG. 19 , the Field of View may include one or more objects1910. The LIDAR sensor may determine the shape and distance of eachobject in the Field of View and transmit that data to the controlsystem. The thermal imaging device may detect difference in temperaturethroughout the Field of View and transmit that data to the controlsystem. The control system may then overlay the thermal imaging data onthe LIDAR scan image to create a 3D temperature profile. As such,temperature gradients 1920 may be visible on the objects. This processmay be iterative whereby the measurements of the same area may beobtained using different Field of Views so as to better capture thedimensionality of the resulting images.

An exemplary method is shown in FIG. 20 . In step 2001, an initial fieldof view for a LIDAR and thermal imaging scans is established for aparticular location (here, Location X). In step 2002, a LIDAR scan isgenerated, and in step 2003, a thermal imaging scan is generated. Instep 2004, both scans are transmitted to a controller in a controlsystem. In step 2005, the thermal scan is overlaid onto the LIDAR scanto provide a 3D view of the thermal imaging data. In order to improvethe quality of the 3D temperature profile, scans may be repeated at thesame or at different field of views (step 2006). This data may beevaluated and combined to create the 3D temperature profile of LocationX (step 2007), These steps may be repeated at select/predetermined timeintervals in order to evaluate the temperature profile of Location Xover time (step 2008). Thus, the temperature of the 3D space of LocationX ma be monitored over time. The 3D temperature profile may be createdat any suitable interval, such as, for example, each millisecond,second, or every 1, 2, 3, 5 or 10 minutes. The length of the timeinterval may depend on the sensitivity of the items in the cold storagespace and the particular temperature profiling system used, among otherfactors.

In some cases, in order to better visualize the change in temperature atcertain locations, “temperature sensitive materials,” or TSM, which arematerials having a low specific heat capacity, may be added to portionsof the warehouse structures, including on walls, scaffolding, pallets,and the like. The low specific heat materials change temperaturerelatively easily and so smaller changes in heat may more greatly affectthe temperature of such materials. Thermal imaging devices measures theamount of infrared radiation emitted from an object, and an increase intemperature will increase the amount of emitted radiation that thethermal imaging sensors may detect. Thus, materials that changetemperature more easily make it easier for the scans to determinesmaller fluctuations in heat at a particular location. This addition ofTSM into portions of the warehouse, pallets, and the like, may beparticularly useful when the product packaging has a high specific heatand so does not significantly register changes in heat fluctuations inthe warehouse. FIG. 21 illustrates a portion of a cold storage warehouse2100 that includes scaffolding 2110 and pallets 2120 and stored items2101. The TSM 2125 may be present on the scaffolding 2110, the pallets2120, on other surfaces 2130, and/or suspended 2140 in the air.

A number of configurations would allow for the LIDAR and thermal imagingscans to be obtained throughout the Facility. LIDAR and thermal imagingdevices may be placed throughout the Facility in addition to or as partof a Node or Smart Device. The information from each scan may betransmitted via Nodes, Smart Devices, or other transceivers to theenvironmental control system so that the 3D temperature profiles may begenerated and combined to provide a 3D profile of the space as a whole.Since sonic of the transmission and reception frequencies may overlap,may be close to overlapping or may have harmonics or subharmonics thatoverlap with other base transmission frequencies, it may be desirablethat a transmission controller may multiplex the signals to the variouscomponents to ensure that the transmissions do not interfere with eachother.

The 3D temperature (or other environmental parameter) profile allowsevery part of the Facility (that is monitored using the system) to havea temperature history. Using the methods described above using physicaland virtual tags, the location of each object, including products, canbe determined over time. Combining the two sets of information, theEnvironmental Control System can determine the environmental conditionof each object over time.

While the temperature control system may be used to track thetemperature of each product as it proceeds through its storagelifecycle, the temperature control system may also have additionalfeatures that allow for the control system to optimize cooling withinthe cold storage space, For example, the 3D temperature profile of theFacility, or a portion of the same, may be monitored before, after, orduring certain Action Events, which are actions in or concerning theFacility that may have an effect on the environmental conditions in theFacility.

In sonic embodiments, the environmental control system identifies anAction Event related to the Facility and correlates the Action Eventwith an increase or decrease in temperature in one or more areas (alsoreferred to as zones) in the space. For example, the environmentalcontrol system may be in electronic communication with a sensor at a baydoor and determine via the sensor that the bay door has been opened. Theenvironmental control system may monitor changes in that area, orthroughout the Facility, to assess the effect of the bay door opening onenvironmental conditions in the Facility, As another example, theenvironmental control system may compare changes in humidity in theFacility based on the type or amount of goods being stored at a specifictime. Correlating Action Events with changes in the environment in theFacility may allow for optimization of heating, cooling, or other cyclesin the Facility. In addition, the HVAC system may intermittently runcycles in the cold storage space for heating/cooling the room. Thetemperature control system may be in electronic communication with theHVAC system to determine that effect of any particular cycle or time ofcycling. Such correlation may allow for optimization of timing andcycling of the HVAC system. Another Action Event is a change in theposition of pallets. A concentration of colder items in a particularspace may decrease the temperature near those items, or theconfiguration may change the air flow in the space.

In some embodiments of the invention, if a temperature in a particulararea of the Facility exceeds a certain temperature, the temperaturecontrol system will initiate an influx of cooler (lower temperature) airin or near that area. The amount and position of the influx of coolerair may depend on both magnitude of temperature increase and thelocation of the temperature increase. For example, an influx of coolerair may be provided directly to a position within or adjacent to thedetected warmer temperature. Alternatively, because heat rises, theinflux of cooler air may be provided to an area above where the increasein temperature is located. The temperature control system mayalternatively or additionally extract (via vacuum or other methods) aportion of air (e.g., a portion of warmer air) in the room. Again, thesystem may extract air in the area of the warehouse that has thetemperature above a certain threshold or may extract air in anotherportion of the cold storage space. This may likewise be performed if atemperature is too low, whereby an influx of warmer air may be providedto the area or cooler air may be extracted. The environmental monitoringmay allow for experimentation as to where the influx of air is mostefficient, and where a return duct is best located. Additionally,determining where to locate of fans in the Facility may be determined.

In addition to analysis and optimization, the environmental controlsystem may identify problematic areas so that timely action may be takento protect product quality. For example, if a temperature exceeds acertain temperature (or, for example, moisture levels exceed a certainamount), an alarm may be generated to at least one Smart Device or Node.Alarms include both visual alarms (such as on user interface) andaudible alarms. In sonic cases, an alarm may be generated if a certainarea exceeds a specific temperature for more than a predetermined numberof minutes. Other environmental parameters falling outside predeterminedranges, such as light or vibration, may also act to initiate alarms.

Using Location and Environmental Control Systems in Cold StorageApplications

As described above, the location and position of any Agent or item inthe Cold Storage Facility can be known provided that a physical tag(e.g., Node or Smart Device) and/or Virtual Tag is correlated with each.Additionally, the environmental history of the Agent or item can bedetermined according to methods described herein. Such use of locationand environmental tracking will now be described to explain how thelocation and environmental conditions of products may be monitoredthroughout their storage life cycle in the Facility.

First, the arrival of a product to a cold storage facility will bedescribed. A delivery driver may locate the Property on which the ColdStorage Facility sits and be routed to the Property using conventionalmethods such as GPS. Once approaching the facility, more specificpositioning methods may be used to identify the appropriate location fordrop off of the contents (e.g., one or more pallets of products).Referring to FIG. 22 , a delivery truck 2200 may approach a Cold StorageFacility 2202, which has a series of possible delivery locations thereinsuch as delivery bays 2204-2212. The driver or truck may have a physicallocating tag including a Node 2214 to identify a particular truck, whichmay thus, identify a particular delivery of goods. In some embodiments,the Node may be associated with a unique ID that can further identifythe truck, driver, and/or products. The Node 2214 may be used to trackthe driver's route to the Cold Storage Facility 2202. Additionally, asthe delivery truck 2200 approaches the Facility 2202, the Node 2214 maycommunicate with one or more Nodes 2216 at (or in communication with)the facility, which may thus guide the driver in the delivery truck 2200via a Smart Device or a heads-up display 2218 to the correct locationfor unloading the particular products. In some cases, when the truckwith the Node 2214 thereon approaches the Facility 2202, the unique IDfrom the driver/truck provides a permission for the truck to enter thepremises and/or provides coordinates or directions to the Smart Deviceor heads up display 2218 of the driver to provide authorization anddirections to the appropriate drop off location, such as delivery bay2210. Once arriving or approaching the correct drop off location, thedriver may also transmit to the one or more Nodes 2216 at the warehousea notice that the products are being delivered so that appropriatepersonnel may automatically be dispatched to the correct location (here,delivery bay 2210) to unload or otherwise process the delivery.

Referring to FIG. 23 , once the delivery truck arrives to the correctdelivery bay 2310 at the Facility 2302, personnel may be prepared tounload. In some embodiments, the signal sent from the truck's Node tothe warehouse Node identified the products to be unloaded at thedelivery bay 2310. Thus, an Employee/Agent 2322 at the delivery bay 2310may have a Smart Device or heads up display 2318 that identified theproducts to be unloaded. Each pallet 2324, and in some cases, eachproduct 2326, on the delivery truck 2300 may have a Node 2314 orphysical tag thereon identifying the product and providing anymanufacturing, history, or other data about the product 2326. Examplesof physical tags 2314 include RFID, hash or barcode labels. The physicaltag 2314 may be assigned to the pallets 2324 and/or products 2326thereon. Such physical tags 2314 may be scanned by the Employee 2322 toconfirm the identity of the products 2326 and such information may betransmitted to one or more Nodes 2316. The Employee may further create avirtual tag 2320 identifying the pallet 2324 and/or product 2326,wherein the virtual tag 2320 identifies the location of the pallets 2324and products 2326 at that time and transmits the information to the oneor more Nodes 2316. Additionally, the Smart Device 2318 may interactwith one or more Nodes in the delivery truck to ascertain thetemperature of the storage container in the truck throughout delivery,or at delivery. This temperature information may then be attached to theproducts during their storage in the Cold Storage Facility as an initialtemperature.

The pallets 2324 and/or products 2326 may then be transported to asecondary location within the Cold Storage Facility 2302, for examplevia a transport (not shown) such as the forklift. The orienteeringmethods described herein may direct the driver of the transport to theappropriate location within the Cold Storage Facility 2302 using, forexample, a Smart Device 2318 and/or a heads-up display (not shown). Whena pallet 2324 or product 2326 is loaded onto a transport, a Smart Device2318 on the transport may identify the pallet 2324 or product 2326 basedon identifying the Virtual Tag 2300 in its Field of View, interactingwith a Node on a pallet or on the delivery truck, or by scanning aphysical tag 2314. The Smart Device 2318 on the transport may thencorrelate the movements of that product/pallet 2324/2326 with that ofthe transport as it moves throughout the Facility 2302 until theproduct/pallet 2324/2326 is relocated to a second location. The SmartDevice 2318 may transmit the information about the pallet/product2324/2326 as it is transported through the Facility 2302 to theWarehouse Management System or a separate database, thereby providinginformation about where the pallet/product 2324/2326 moved through theFacility 2302 and the environmental conditions throughout, as will bediscussed in further detail below. The Smart Device 2318 on thetransport may also include an accelerometer or vibration sensors thereonso that any jarring movements/dropping of products during transport isalso detected and transmitted to the appropriate database or VMS.

When a transport moves through the warehouse space with a Node, it maypass through different temperature zones within the Facility 2302.However, the transport's Node may “pick up” the temperature or otherenvironmental conditions of each zone in a warehouse as it passesthrough the building. For example, a transport may move a pallet/product2324/2326 from its original location and update temperature conditionsas it moves throughout the building, such as by correlating the locationwith the temperature periodically at defined intervals) such as after acertain time period or after the transport has moved a certain distance,The Node in the transport may transmit a pallet/product 2324/2326location and transmit it to the control system, whereby its location canbe correlated with the temperature at that location and time. This canbe repeated sequentially and preferably automatically as the transportmoves through the Facility 2302. Alternatively, the Facility 2302 may bedivided into certain temperature zones and once the transport leaves aparticular zone (based on the orienteering methods or GPS methods), thetemperature of the pallet/product 2324/2326 updates to that temperatureidentified within the new zone. This could be updated at an interval andthe temperature zones could be very small (less than 5 feet) or larger(such as every 5-10, 10-20, 30-40, and 50 or more feet) depending on theparticular system and warehouse.

When the pallet/product arrives at a second (or third, etc.) location, anew virtual tag may be created at that location by a Smart Device, e.g.,a smart device on the transport. The new coordinates of the productswith the identifying information are transmitted to the appropriatedatabase. Thus, the environmental conditions at those coordinates willbe associated with the products at that location for the time that thepallet/product is at that location.

Referring to FIG. 24 , once the operator of the transport 2428 locatesthe desired location of the pallet/product 2424/2426, a virtual tag 2420may be created by the Employee or Agent at that location. In some cases,a virtual tag 2420 may be automatically generated upon somepredetermined condition. For example, the pallet 2424 or transport 2428may have an accelerometer, barometric pressure sensor, or other sensorwhich determines when the pallet 2424 or product 2426 has been set inthe desired location, or even just picked up or set down. When thecondition is met, a virtual tag 2420 may be created. In someembodiments, a position or pattern recognition sensor (e.g., LIDAR)recognizes a position of the pallet either via the configuration ofitems at that location or via a physical tag scanned at the location.Thus, when the LIDAR scanner detects a certain pattern or physical tag,a new virtual tag 2420 is automatically created and transmitted to theat least one Node 2416. The information from the pallet 2426 iscommunicated associated with the virtual tag 2420 so that the virtualtag 2420 contains information about the product, including for example,the history of where the product 2426 has been and, in some cases, itsprior environmental history, within the Facility 2402 and/or duringtransport and/or manufacture. This newly generated virtual tag 2420 canoverwrite the virtual tag 2402 associated with the particular palletposition or a separate virtual tag 2402 may be present. Thus, eachpallet 2424 and/or product 2426 may be located within the Facility 2402at any given time and the location and environmental history of wherethe product 2426 has been can be ascertained. Traditional methods ofinventory management such as RFID or scanning technology may also beused in combination with the virtual tags in some cases, for example, toconfirm accuracy of the virtual taps. Additionally, Nodes may be presentat each pallet position, and the Smart Device may transmit informationabout the products to the Node, which may then be transmitted to thecontrol system, VMS, or other database.

One method according to an embodiment of the invention is shown in FIG.25 , wherein step 2501 describes generating a virtual tag with a palletinformation at a first pallet location. Next, in step 2502, the locationof the pallet may be correlated with the 3D temperature profile at thefirst pallet location to determine the temperature history of the palletwhile at the first pallet location, and this information may be added toa database about the products in the pallet. Next, in step 2503, thepallet may be moved to a second pallet location, and a new virtual tagwith the pallet information may be generated at the second palletlocation. Finally, in step 2504, the location of the pallet at thesecond pallet location may be correlated with the 3D temperature profileat the second pallet location to determine the temperature history ofthe pallet while at the second pallet location, and this data may beadded to the database about the products in the pallet. Thus, thetemperature history of the products in that pallet may be known. Foradditional information on the products' environmental history, thelocation and environmental conditions of the pallet/product duringtransport may also be determined as described above.

When the products are to be moved to another location, the same processof tracking the products during transport and storage described abovemay be used. Mixing of products and pallets may also be possible wherethe specific products are identified as they are mixed. For example,referring to FIG. 26 , a first pallet 2610 may have 12 products (“A”)thereon and a second pallet 2620 may also have 12 products (“B”)thereon. The products A and B may be the same or different type ofproducts. The pallets may then be mixed so that each new pallet 2630,2640 has 6 products of A and 6 products of B thereon.

Originally, each pallet may be tracked through the warehouse via thecombination of virtual tags and association with a transmitter/receiveron a transport (or other means of tracking the movements of theproducts) as discussed above. As such, the pallet 2610 products have alocation and environmental history associated therewith, and the pallet2620 products have a location and environmental history associatedtherewith. While mixing the pallets, each product may be scanned andassigned to a new pallet. The controller or other database processor maythen assign a location and environmental history to each product in thenew pallet. Thus, in this example, the database would include alocation/environmental history for the A products and alocation/environmental history the B products while in the separatepallets 2610, 2620. Thereafter, the A and B products will have the samelocation and environmental data (while in the same pallet such as pallet2630) but each product A in pallet 2630 will have a database listingthat includes the location and environmental conditions for pallet 2610prior to mixing and then the location and environmental conditions forpallet 2630 after mixing. Thus, environmental tracking may be possibleeven with pallet mixing.

In some embodiments, a product may be tracked from manufacture to finalsale through the systems and methods. Once a product is manufactured,the product may be assigned position tags (virtual or physical) thatidentify the location of the products while in the manufacturingfacility. Then, the products may be tracked during transport through thefacility by being associated with a transport Node or Smart Device. Oncethe products are placed onto a delivery truck or other means oftransportation, the location and environmental conditions of thetruck/transport may be known by GPS or other location trackingtechnologies (location), and via sensors in the transport (environmentalconditions) and the information may be transmitted to a database by aNode or Smart Device on the transport. Then, processing and storage atthe Cold Storage Facility may be tracked as discussed above. Once readyto distribute the products at a sales location, the product may betracked through the Cold Storage Facility to a truck/transport, asdescribed above, and then tracked during transportation, as furtherdescribed above. If desired, the product can continue to be tracked atthe sales location. Physical and/or virtual tags may be created in thestore (at one or multiple locations) to identify the location of theproduct in the store. Thus, from manufacture to final sale, the locationand environmental conditions of a product may be known. This may bebeneficial for product tracking and quality assurance efforts.

As with the products and pallets, Employees be tracked with Nodes orSmart Devices carried on the Employee. The movements of the Employee maybe tracked throughout the Cold Storage Facility. Additionally, theEmployee's access to particular areas of the Facility may be based ontransmissions from a Node, Smart Device, or transmitter on the Employee,and if the Employee is not in the appropriate area of the warehouse, analarm or notification may be generated. Additionally, one or moreEmployees may have a temperature or other environmental sensor onhis/her person, including on a hat or electronic device carried with theperson, and the temperature of the air at the employee's location overtime may also be tracked to provide further information about thetemperature in the Facility. Tracking the location of each employee withthe transmitter, Node, or Smart Device may also allow for countingand/or locating employees in the event of an emergency. Positionalsensors on the Employee's hat (e.g., a sensor on the front or back) mayalso be useful to provide information about the employee's orientationfor the purpose of orienteering or direction of interest as describedherein.

User Interfaces

Provided according to embodiments of the invention are user interfacesthat may be used with the methods, devices, and apparatus describedherein. A user interface may displayed on a tablet, Smart Device, orother flat screen, or in some embodiments be presented in a virtualreality environment, such as via a virtual reality headset. It should benoted that although a Smart Device is generally operated by a humanuser, some embodiments of the present invention include a controller,accelerometer, data storage medium, Image Capture Device, or an infraredcapture device being available in a handheld or unmanned vehicle orother Agent.

In some embodiments of the invention, a user interface may provide anoverview of operations in the Cold Storage Facility. In some cases, auser interface tracks sensors, pallets, employees, and equipment, suchas via a touch screen. Referring to FIG. 27 , the user interface 2734provides an overview of a room in a Cold Storage Facility, and thepallets 2724, sensors 2727, transports 2728, and employees 2733 locatedin the room, each of which may be identified by the system and userinterface by virtue of a Node, transceiver, or Smart Device associatedwith the item or person. The user interface may have additional screensfor additional rooms, or the entire Facility may be viewed at once. Insome embodiments, different types of personnel may be identified in theuser interface by a different type or different color icon. Differenttypes of sensors may also be identified by a particular color or shape.The remaining icons may also be distinguished for any reason by varyingthe style or color of each icon. In some embodiments, icons in the userinterface may be “clicked” or interacted with in order to provide moreinformation about the item, or to find controls with respect to an item.

For example, in FIG. 27 , each pallet 2724 may be assigned a number.When a user clicks on the icon for a particular pallet, the userinterface may identify the type of product, the time at that location,and an environmental or location history. Furthermore, if a particularpallet is at a location that is deemed to exceed a temperature limit,details regarding the excess temperature may be provided. In such event,the icon may provide an alert to notify such as a sound, blinking icon,or push notification to let the user know of the problem at thatposition. As another example, a sensor 2727 icon may be clicked toobtain a history of data obtained by the sensor. Additionally, if thesensor is detecting a parameter that is outside of the allowed range forthe parameter, the user interface may provide an alert as describedabove. The alerts may also be useful for use with employees, as an alertmay be generated on the user interface if an employee accesses anon-authorized area in the Facility. Additionally, in the event of anemergency, the user interface may be used to locate Employees forevacuation or other emergency measures.

Apparatus

The following section describes various apparatus that may be suitableto perform the methods described herein.

Referring now to FIG. 28 , an automated controller is illustrated thatmay be used to implement various aspects of the present invention invarious embodiments, and for various aspects of the present invention.Controller 2800 may be included in one or more of: a wireless tablet orhandheld smart device, a server, an integrated circuit incorporated intoa Node, appliance, equipment item, machinery or other automation. Thecontroller 2800 includes a processor unit 2802, such as one or moresemiconductor-based processors, coupled to a communication device 2801configured to communicate a communication network (not shown in FIG. 28). The communication device 2801 may be used to communicate, forexample, with one or more online devices, such as a smart device, aNode, personal computer, laptop, or a handheld device.

The processor 2802 is also in communication with a storage device 2803.The storage device 2803 may comprise any appropriate information storagedevice, including combinations of digital storage devices (e.g. an SSD),optical storage devices, and/or semiconductor memory devices such asRandom Access Memory (RAM) devices and Read Only Memory (ROM) devices.

The storage device 2803 can store a software program 2804 withexecutable logic for controlling the processor 2802. The processor 2802performs instructions of the software program 2804, and thereby operatesin accordance with the present invention. The processor 2802 may alsocause the communication device 2801 to transmit information, including,in some instances, timing transmissions, digital data and controlcommands to operate apparatus to implement the processes describedabove. The storage device 2803 can additionally store related data in adatabase 2805 and database 2806, as needed.

Referring now to FIG. 29A, an illustration of an exemplary wireless Node2910 configured with a transceiver 2924 to wirelessly communicate viaone or more wireless communication Modalities, including a bandwidth andprotocol, such as the Bluetooth 5.1;: BLE5.1; UWB, Wi-Fi RT; and/or GPSstandard is illustrated. As discussed, many different Modalities ofwireless technology may be utilized with the content presented herein,but a BLE5.1, “radio” module is an interesting example since itsstandards provide for angle of arrival (AoA) capability as well as angleof departure (AoD) and a distance determination based upon a timingsignal. With AoA/AoD, a designed antenna array 2925 can be used by aradiofrequency transceiver 2924 to measure a phase shift amongstmultiple antenna elements to estimate distance differences between theantennas and to extract an angle from the antenna array to the source ofradiation. A BLE5.1-consistent multichip transceiver 2924 may includecircuitry and software code to perform the acquisition of data anddetermine the angle of arrival in some examples. In other examples, aBLE5.1-consistent multichip transceiver 2924 may control the acquisitionof data from an antenna array while streaming the data to off moduleprocessing capabilities. The BLE5.1-consistent Node 2910 may containfunctional blocks of circuitry for peripheral 2920 control. Theperipherals may include a connection to external host controllers/MCUs2921. The peripheral 2920 control may also interact with peripheral andIoT sensors and other devices 2922.

The BLE5.1-consistent Node 2910 may include a processing element 2923which may have its own memory of different types as well as capabilitiesfor encryption of data. The BLE5.1 consistent Node 2910 may also havetransceiver 2924. This circuitry may include baseband and RF functionsas well as control the AoA functions and the self-verifying arrayfunctions. The Bluetooth communications 2924 may receive signals throughan on-module antenna 2925 or an external antenna or array of antennas maprovide external RF input 2926, The BLE5.1-consistent Node 2910 mayinclude functional circuitry blocks for control of security functions2927, crypto-generations, random number generation and the like, TheBLE5.1-consistent Node 2910 may include functional blocks for powermanagement 2928.

The BLE5.1-consistent Node 2910 may be operative for quantification oftemperature aspects of the Node 2910, battery-control functions andpower-conversion functions. An external power source 2933 may beincluded to provide electrical energy to a power management unit 2928which, in some examples, may be from a battery unit, or a grid connectedpower supply source in other examples. The BLE5.1-consistent Node 2910may include functions for control of timing and triggering 2929. In arelated sense, the BLE5.1-consistent Node 2910 may include function: forclock management 2930 within the module, The BLE5.1-consistent Node 2910may also include circuit elements that are always-on 2931 to allowexternal connections 2932 to interact with the device and perhaps awakeit from a dormant state. There may also be other customized and/orgeneric functions that are included in a BLE5.1-consistent Node 2910and/or multichip module.

Referring now to FIG. 29B, a Node 2910 included in a higher orderdeployment assembly is illustrated. A deployment Node 2950 may be inlogical communication with one or more of: sensors, customized controlcommands, antenna array designs and the like.

A Node 2950 may, include multiple antennas or antenna arrays 2951-2956.As described previously, the Node 2950 may include a transceiver module2910, and in some examples, the transceiver module may includeBluetooth-adherent aspects. Communications received via an antenna2951-2956 may be directly ported into the transceiver module 2910.Embodiments may also include routing particular antenna/antenna arrayoutputs to the transceiver module 2910 in a controlled and timedsequence. A processing module 2970 may coordinate a connection of theNode 2950 to external peripherals.

In some examples, circuitry 2980 to logically communicate with one ormore of: a Peripheral, a data connection, cameras and sensorscontrollers, and components to perform data and image acquisition ofvarious kinds, or it may interface external components with the Node2950.

The Node 2950 may also include its own power management unit 2960 whichmay take connected power or battery power or both and use it to provethe various power needs of the components of the assembly. The Node 2950may have its own processing modules 2970 or collections of differenttypes of processing functions which may have dedicated memory components2971. In some examples, specialized processing chips of various kindssuch as Graphical Processing Units and fast mathematics functioncalculators as well as dedicated artificial intelligence processingchips may be included to allow the Node 2950 to perform variouscomputational functions including location determination of wirelesslyconnected devices amongst other functions. There may be numerous otherfunctions to include in a Node 2950 and alternatives types of devices toperform the functions presented herein.

Referring now to FIG. 30 , a block diagram of an exemplary mobile device3002 is illustrated. The mobile device 3002 comprises an optical capturedevice 3008 to capture an image and convert it to machine-compatibledata, and an optical path 3006, typically a lens, an aperture or animage conduit to convey the image from the rendered document to theoptical capture device 3008. The optical capture device 3008 mayincorporate a CCD, a Complementary Metal Oxide Semiconductor (CMOS)imaging device, or an optical sensor 3024 of another type.

A microphone 3010 and associated circuitry may convert the sound of theenvironment, including spoken words, into machine-compatible signals.Input facilities may exist in the form of buttons, scroll wheels, orother tactile sensors such as touch-pads: in some embodiments, inputfacilities may include a touchscreen display.

Visual feedback to the user is possible through a visual display,touchscreen display, or indicator lights. Audible feedback 3034 may comefrom a loudspeaker or other audio transducer. Tactile feedback may comefrom a vibrate module 3036.

A magnetic force sensor 3037 may sense the magnetic field environment ofthe smart device and support direction determinations.

A motion sensor 3038 and associated circuitry convert the motion of themobile device 3002 into machine-compatible signals. The motion sensor3038 may comprise an accelerometer that may be used to sense measurablephysical acceleration, orientation, vibration, and other movements. Insome embodiments, motion sensor 3038 may include a gyroscope or otherdevice to sense different motions.

A location sensor 3040 and associated circuitry may be used to determinethe location of the device. The location sensor 3040 may detect GPSradio signals from satellites or may also use assisted GPS where themobile device may use a cellular network to decrease the time necessaryto determine location. In some embodiments, the location sensor 3040 mayuse radio waves to determine the distance from known radio sources suchas cellular towers to determine the location of the mobile device 3002.In some embodiments these radio signals may be used in addition to GPS.

The mobile device 3002 comprises logic 3026 to interact with the variousother components, possibly processing the received signals intodifferent formats and/or interpretations. Logic 3026 may be operable toread and write data and program instructions stored in associatedstorage or memory 3030 such as RAM, ROM, flash, or other suitablememory. It may read a time signal from the clock unit 3028. In someembodiments, the mobile device 3002 may have an on-board power supply3032. In other embodiments, the mobile device 3002 may be powered from atethered connection to another device, such as a Universal Serial Bus(USB) connection.

The mobile device 3002 also includes a network interface 3016 tocommunicate data to a network and/or an associated computing device.Network interface 3016 may provide two-way data communication. Forexample, network interface 3016 may operate according to the internetprotocol. As another example, network interface 3016 may be a local areanetwork (LAN) card allowing a data communication connection to acompatible LAN. As another example, network interface 3016 may be acellular antenna and associated circuitry which may allow the mobiledevice to communicate over standard wireless data communicationnetworks. In some implementations, network interface 3016 may include aUniversal Serial Bus (USB) to supply power or transmit data. In someembodiments, other wireless links may also be implemented.

As an example of one use of mobile device 3002, a reader may scan somecoded information from a location marker in a Structure with the mobiledevice 3002. The coded information may be included on apparatus such asa hash code, bar code, RFID or other data storage device. In someembodiments, the scan may include a bit-mapped image via the opticalcapture device 3008. Logic 3026 causes the bit-mapped image to be storedin memory 3030 with an associated time-stamp read from the dock unit3028. Logic 3026 may also perform optical character recognition (OCR) orother post-scan processing on the bit-mapped image to convert it totext. The reader may then upload the bit-mapped image (or text or othersignature, if post-scan processing has been performed by logic 3026) toan associated computer via network interface 3016.

As an example of another use of mobile device 3002, a reader may capturesome text from an article as an audio file by using microphone 3010 asan acoustic capture port. Logic 3026 causes audio file to be stored inmemory 3030. Logic 3026 may also perform voice recognition or otherpost-scan processing on the audio file to convert it to text. As above,the reader may then upload the audio file (or text produced by post-scanprocessing performed by logic 3026) to an associated computer vianetwork interface 3016.

A directional sensor 3041 may also be incorporated into the mobiledevice 3002. The directional sensor may be a compass and produce databased upon a magnetic reading or based upon network settings.

A LiDAR sensing system 3051 may also be incorporated into Smart Device3002. The LiDAR system may include a scannable laser light (or othercollimated) light source which may operate at nonvisible wavelengthssuch as in the infrared. An associated sensor device, sensitive to thelight of emission may be included in the system to record time andstrength of returned signal that is reflected off of surfaces in theenvironment of Smart Device 3002. Aspects relating to capturing datawith LiDAR and comparing it to a library of stored data (which may beobtained at multiple angles to improve accuracy) are discussed above.

Conclusion

Various embodiments of the present invention may include one or moresystems including a controller and electronic and/or solid state sensorsand electromechanical and optical devices can be configured to performparticular operations or actions by virtue of having software, firmware,hardware, or a combination of them installed on the system that inoperation causes or cause the system to perform the actions. One or morecomputer programs can be configured to perform particular operations oractions by virtue of including instructions that, when executed by dataprocessing apparatus, cause the apparatus to perform the actions.

One general aspect includes a method of monitoring a cold storagestructure including: generating a representation of a surface topographywithin a specified area within a structure. The method of monitoringalso includes wirelessly communicating between a transceiver collocatedwith an item in the specified area and one or more reference pointtransceivers.

In some embodiments, the method of monitoring also includes based uponthe wirelessly communicating between a transceiver collocated with anitem in the specified area and one or more reference point transceivers,generating positional coordinates for the items with the specified areawith the structure. The method of monitoring may also include with athermal energy detection apparatus, quantifying amounts of thermalenergy at disparate positions included in the specified area within thestructure. The method of monitoring also includes generating a userinterface including a 3d temperature profile of the items within aspecified area within the structure, the user interface including atleast portions of the representation of the surface topography and thequantified amounts of thermal energy at disparate positions included inthe specified area within the structure. The method of monitoring alsoincludes at selected time intervals repeat the step of quantifyingamounts of thermal energy at disparate positions included in thespecified area within the structure. The method of monitoring alsoincludes with a controller monitoring the thermal energy at disparatepositions included in the specified area within the structure for anincrease or a decrease in thermal energy outside a predeterminedtemperature range; and if an amount of thermal energy at the disparatepositions included in the specified area within the structure falloutside the predetermined temperature range, with a controller initiateone or both of an influx of air to a predetermined position in thestructure or initiate an extraction of air from a predetermined positionin the structure. Other embodiments of this aspect include correspondingcomputer systems, apparatus, and computer programs recorded on one ormore computer storage devices, each configured to perform the actions ofthe methods.

Implementations may include one or more of the following features. Themethod where the step of quantifying an amount of thermal energyincludes quantifying infrared energy as a digital value using amicrobolometer. The method where the sensor quantifying a thermal stateincludes operation of an electronic temperature sensing device includingat least one of thermocouple, a thermistor, and a resistance basedsensor. The method where if amounts of thermal energy quantified includea temperature greater than the predetermined temperature range,initiating with the controller one or both of: an influx of air with alower amount of thermal energy than the amount quantified; andextracting air with an amount of thermal energy higher than thepredetermined temperature range. The method where the influx of air isprovided to a location of the increase in temperature. The method wherean influx of air is provided to an airflow vent above a locationincluding thermal energy higher than the predetermined temperaturerange, as determined by the step of quantifying amounts of thermalenergy. The method where if the thermal energy at a location is lessthan the predetermined temperature range, initiating with the controllerone or both of an influx of air with a higher amount of thermal energythan the amount quantified; and extracting air with an amount of thermalenergy lower than the predetermined temperature range. The method wherethe influx of air is provided to the location of the increase intemperature.

In some embodiments, implementations may include one or more of thefollowing aspects, identifying action events related to the cold storagestructure and the controller correlates the action events with anincrease or decrease in temperature outside of the predeterminedtemperature range. The method where the action event is one or both of:an opening of a door and a closing of the door in the cold storagestructure. The method where the action event is one of a heating cycleand a cooling cycle by a HVAC unit. The method additionally includingthe steps of: with a lidar unit in electronic communication with thecontroller, generating positional data points of items in the area ofthe structure; and with the controller creating a 3d position map of thecold storage structure at selected intervals to identify objects in thecold storage structure. The method where the step of quantifying amountsof thermal energy includes measuring a temperature of an item in thespecified area within the structure.

The method additionally may include the step of: with the controllercorrelating a change in temperature of the item in the specified areawithin the structure. The method additionally including the step of withthe controller determining at least one of: a change in a position ofthe item in the specified area within the structure, and removal of theitem from the specified area within the structure. The method where theitem is a worker in the cold storage structure.

Still other aspects of some embodiments may include measuring an amountof moisture in the specified area with a humidity detection apparatus inelectronic communication with the controller. The method additionallyincluding the step of initiating an alarm if the amount of moisture inthe specified area is outside a predetermined humidity range. The methodwhere the wirelessly communicating between a transceiver collocated withan item in the specified area and one or more reference pointtransceivers includes a ultra-wideband transceiving. The method wherethe wirelessly communicating between a transceiver collocated with anitem in the specified area and one or more reference point transceiversincludes a bluetooth transceiving. Implementations of the describedtechniques may include hardware, a method or process, or computersoftware on a computer-accessible medium.

Certain features that are described in this specification in the contextof separate embodiments can also be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment can also be implemented incombination in multiple embodiments separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Moreover, the separation of various system components in the embodimentsdescribed above should not be understood as requiring such separation inall embodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order show, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous. Nevertheless, it will be understood thatvarious modifications may be made without departing from the spirit andscope of the claimed disclosure.

What is claimed is:
 1. A method for providing digital content associated with a wireless node, the method comprising the steps of: a) representing a position of the wireless node as positional coordinates; b) associating digital content stored with the positional coordinates of the node; c) establishing a radio target area for an energy receiving sensor, said radio target area comprising an area in a field of view of a charged couple device comprising the location of the node; d) receiving energy into the energy receiving sensor from the radio target area; e) generating a digital representation of the energy received into the energy receiving sensor at an instance in time; f) generating a user interactive interface comprising a digital representation of the energy received into the energy receiving sensor at the instance in time and a dynamic portion representative of the location of the node; g) receiving a user input into the dynamic portion of the user interactive interface; and h) based upon the user input received into the dynamic portion of the user interactive interface, displaying the digital content associated with the positional coordinates of the node in the user interactive interface.
 2. The method of claim 1 wherein the user interactive interface is displayed on a smart device, and the method additionally comprises the steps of receiving into the smart device a digital quantification of a condition by an automated sensor within the radio target area at the instance in time, said digital quantification generated with a sensor; and displaying the digital quantification of the condition in the user interactive interface.
 3. The method of claim 2 additionally comprising the steps of: receiving multiple disparate energy wavelengths into the energy receiving sensor at the instance in time, the disparate energy wavelengths received from different geospatial locations; and indicating the disparate energy wavelengths in the user interactive interface.
 4. The method of claim 2 wherein the node comprises a transceiver capable of wireless communication between the node and at least one reference point transceiver using a Bluetooth modality; and the method additionally comprises the step of generating positional coordinates for the node based upon the wireless communication between the node and the at least one reference point transceiver.
 5. The method of claim 4 additionally comprising the step of identifying a wireless communication area comprising the radio target area of the energy receiving sensor, the wireless communication area based upon a communication distance of a Bluetooth modality in an area encompassing the energy receiving sensor.
 6. The method of claim 5 wherein the positional coordinates comprise Cartesian Coordinates.
 7. The method of claim 5 wherein the positional coordinates comprise at least one of: an angle of arrival and an angle of departure.
 8. The method of claim 2 additionally comprising the steps of a) receiving into the smart device a digital quantification of a temperature within the radio target area at multiple disparate instances in time; b) with a controller, monitoring the temperature within the radio target area for an increase or a decrease in thermal energy outside a predetermined temperature range; and c) if a temperature within the radio target area is monitored outside the predetermined temperature range, with a controller initiate one or both of: an influx of air to a predetermined position in the radio target area or initiate an extraction of air from a predetermined position in the radio target area.
 9. The method of claim 8 wherein the influx of air to a predetermined position comprises air at a lower temperature than the predetermined temperature range.
 10. The method of claim 2 wherein the radio target area comprises a cold storage.
 11. The method of claim 2 wherein the radio target area comprises the position of the node, the position of the automated sensor and ambient electromagnetic energy to the energy receiving sensor.
 12. The method of claim 11 wherein the radio target area further comprises a frustum extending outward the energy receiving sensor.
 13. The method of claim 12 additionally comprising the steps of changing a direction of the energy receiving sensor; and redefining the radio target area based upon the change in direction of the energy receiving sensor.
 14. The method of claim 1 wherein the digital content comprises a virtual tag associated with an area comprising the positional coordinates.
 15. The method of claim 1 wherein the node comprises a transceiver capable of wireless communication and the method additionally comprises the steps of: transceiving a wireless communication between the node and multiple reference point transceivers; and generating positional coordinates for the node based upon the wireless communication between the node and the multiple reference transceivers.
 16. The method of claim 15 additionally comprising the steps of: transceiving a wireless communication between the node and multiple reference point transceivers using an Ultra Wideband modality; and generating the positional coordinates based upon the wireless communication between the node and the multiple reference point transceivers.
 17. The method of claim 16 additionally comprising the step of identifying a wireless communication area comprising the radio target area of the energy receiving sensor, the wireless communication area based upon a communication distance of the Ultra Wideband modality in an area encompassing the energy receiving sensor.
 18. The method of claim 1 additionally comprising the step of quantifying an amount of infrared energy as a digital value using a microbolometer.
 19. The method of claim 1 additionally comprising the step of quantifying a thermal state via operation of an electronic temperature sensing device comprising at least one of: a thermocouple, a thermistor, and a resistance based sensor.
 20. The method of claim 19 wherein the energy received into the energy receiving sensor comprises one or more of: an infrared wavelength; and a wavelength between 400 nanometers and 700 nanometers. 